ExGitEngine.JobLimiter (ex_git_engine v0.12.0)

Copy Markdown

Runs long jobs (packs, blame, ahead/behind counts, pushed pack indexing, per-file patches) at most max_jobs at a time, outside the processes that ask for them.

These jobs call dirty NIFs, and the VM has only as many dirty CPU schedulers as cores, shared with every other NIF. Without a limit, a burst of blames on a big repository would take all of them and stall every other diff. The default max_jobs is one less than the dirty CPU schedulers, so one is always free. Set it with config :ex_git_engine, max_jobs: n.

Waiting jobs are dropped when their caller has exited before they start, so a page the user left does not still cost a blame.

Like a busy port, the limiter turns work away when its queue reaches high_water (default 10 * max_jobs): callers get {:error, :busy} at once instead of waiting until they time out. It accepts work again once the queue has drained to low_water (default high_water / 2). Both can be set with config :ex_git_engine, high_water: n, low_water: n.

Started by ExGitEngine.Application. GitAgent sends its long operations here; run/3 is the same for any other function:

ExGitEngine.JobLimiter.run(fn -> expensive() end)

Summary

Functions

Returns a specification to start this module under a supervisor.

Returns the number of jobs waiting for a slot.

Runs fun once a slot is free and returns its result. A job that raises returns {:error, exception}; {:error, :busy} means the queue is full.

Returns the number of jobs running.

Starts a limiter. Options: :name (default ExGitEngine.JobLimiter, nil for none), :max_jobs, :high_water and :low_water.

Like run/3, but answers from (a GenServer.from() received by another server) instead of the caller, so that server can return {:noreply, ...}.

Types

server()

@type server() :: GenServer.server()

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

queued(server \\ __MODULE__)

@spec queued(server()) :: non_neg_integer()

Returns the number of jobs waiting for a slot.

run(server \\ __MODULE__, fun, timeout \\ :infinity)

@spec run(server(), (-> result), timeout()) :: result | {:error, :busy | term()}
when result: term()

Runs fun once a slot is free and returns its result. A job that raises returns {:error, exception}; {:error, :busy} means the queue is full.

running(server \\ __MODULE__)

@spec running(server()) :: non_neg_integer()

Returns the number of jobs running.

start_link(opts \\ [])

@spec start_link(keyword()) :: GenServer.on_start()

Starts a limiter. Options: :name (default ExGitEngine.JobLimiter, nil for none), :max_jobs, :high_water and :low_water.

submit(server \\ __MODULE__, from, fun)

@spec submit(server(), GenServer.from(), (-> term())) :: :ok

Like run/3, but answers from (a GenServer.from() received by another server) instead of the caller, so that server can return {:noreply, ...}.