Per-queue feeder + executor loop. Backpressure-driven: it claims work
from the database into a small in-memory buffer, then spawns at most
concurrency supervised Tasks at a time, draining the buffer as slots free.
Aggressiveness knobs
Throughput is not capped by poll_interval — a finished Task immediately
refills from the buffer (no round-trip) or picks a fresh batch. The poll timer
only governs how promptly newly-inserted work is discovered while idle. The
knobs let a deployment trade DB chatter, latency, and fairness:
concurrency— max Tasks running at once (the executor width).prefetch— extra rows to claim and hold in the buffer beyond the running slots.0(default) means no over-fetch: identical to picking exactly the free slots. Raising it batches picks and absorbs completion bursts, at the cost of cross-node fairness and priority freshness (claimed rows are invisible to other nodes), and buffered rows carry their pick-time inbox snapshot (ctx.all/ctx.childsare batch-loaded with the pick). Buffered rows are heartbeated, so their leases never go stale regardless of depth — depth is decoupled fromlease_ttl.min_demand— batch gate: don't pick unless at least this many slots are free (so picks come fat, not one row at a time). Ignored when the queue is fully idle, to avoid starvation.poll_interval— base idle poll.max_poll_interval— backoff ceiling: an empty pick on a fully idle queue doubles the interval up to this cap, then a non-empty pick (or any in-flight work) snaps back to the base. This is the lever that cuts idle DB load. (LISTEN/NOTIFY is banned, so polling is the discovery path — the point is to poll adaptively, not constantly.)
Inserts, signal wakes, and fan-out transitions poke the queue's schedulers
directly (see GenDurable.Poke): a row that just became runnable is
discovered immediately, not on the next poll tick — same-node always,
cross-node with the :cluster or {:redis, _} transports. Polling covers
what a poke cannot see — retry backoffs, the reaper's wakes, and remote
events under the default :local transport.
Buffered and in-flight rows are both executing + leased; the heartbeat (one
batched UPDATE per tick over buffer ++ in_flight) keeps every claimed row
alive. If this scheduler dies, the whole claimed set waits one lease_ttl for
the reaper — so deeper buffers mean a larger crash blip, bounded by the (short)
TTL, not by the buffer depth.
concurrency_key serialization is enforced at claim time by the database
(the UNIQUE partial index over executing keys): the pick either commits the
only executing row for a key or retries on the violation — no per-step locks,
no pinned connections. See GenDurable.Queries (the pick).
Summary
Functions
Returns a specification to start this module under a supervisor.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.