Per-queue feeder + executor loop (spec §6). Backpressure-driven: it claims work
from the database into a small in-memory buffer, then spawns at most
concurrency supervised Tasks at a time, draining the buffer as slots free.
Aggressiveness knobs
Throughput is not capped by poll_interval — a finished Task immediately
refills from the buffer (no round-trip) or picks a fresh batch. The poll timer
only governs how promptly newly-inserted work is discovered while idle. The
knobs let a deployment trade DB chatter, latency, and fairness:
concurrency— max Tasks running at once (the executor width).prefetch— extra rows to claim and hold in the buffer beyond the running slots.0(default) means no over-fetch: identical to picking exactly the free slots. Raising it batches picks and absorbs completion bursts, at the cost of cross-node fairness and priority freshness (claimed rows are invisible to other nodes). Buffered rows are heartbeated, so they never go stale regardless of depth — depth is decoupled fromlease_ttl.min_demand— batch gate: don't pick unless at least this many slots are free (so picks come fat, not one row at a time). Ignored when the queue is fully idle, to avoid starvation.poll_interval— base idle poll.max_poll_interval— backoff ceiling: an empty pick on a fully idle queue doubles the interval up to this cap, then a non-empty pick (or any in-flight work) snaps back to the base. This is the lever that cuts idle DB load. (LISTEN/NOTIFY is banned, so polling is the only discovery path — the point is to poll adaptively, not constantly.)
Buffered and in-flight rows are both executing + leased; the heartbeat (one
batched UPDATE per tick over buffer ++ in_flight) keeps every claimed row
alive. If this scheduler dies, the whole claimed set waits one lease_ttl for
the reaper — so deeper buffers mean a larger crash blip, bounded by the (short)
TTL, not by the buffer depth.
partition_key serialization (spec §6) is handled per-job at execution time: a
session-level advisory lock is taken on a checked-out connection held for the
whole step, and released after the outcome commits. If the lock is contended,
the row is returned to runnable and skipped.
Summary
Functions
Returns a specification to start this module under a supervisor.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.