Concurrency keys

A concurrency_key is a semaphore of size K on a key: at most K instances sharing the key execute at once, while different keys run in parallel. K comes from the concurrency_limits: engine config, matched by the key's name (the part before :); an unconfigured key defaults to K = 1 — strict mutual exclusion, the answer to "process everything for one account one-at-a-time, but accounts in parallel."

# mutual exclusion (no config for "account" ⇒ K = 1)
GenDurable.insert(Account.Sync, state: %{id: 7}, concurrency_key: "account:7")
GenDurable.insert(Account.Sync, state: %{id: 7}, concurrency_key: "account:7")  # waits for the first
GenDurable.insert(Account.Sync, state: %{id: 9}, concurrency_key: "account:9")  # runs in parallel

# a concurrency GATE: cap the in-flight work against an external API, cluster-wide
# engine: concurrency_limits: [stripe: [limit: 100]]
GenDurable.insert(Charge, state: %{...}, concurrency_key: {:stripe, tenant_id})

Unlike a rate limit (a flow cap — starts per second, blind to how long steps run), a concurrency gate is a stock cap: the slot is held for the whole step and returned on its outcome. A hundred-per-minute rate limit happily launches 100 thirty-second calls at once; limit: 10 never has more than 10 in flight.

How it is enforced

• K = 1 (unconfigured): a unique partial index over executing keys — a second executing row per key is uncommittable. The claim is the lock: held exactly for the step window, released by any outcome, or by the reaper if the worker dies. No per-step locks, no pinned connections.
• Gates (configured): per-key slot counters with a database CHECK — over-admission is uncommittable. The pick debits slots in one batched pass; every outcome credits its slot back. Counters are minted lazily, pre-debited, by the first claim itself — a cold gate (first use, or swept by GC after the key went idle) admits with zero lag. Crash paths under-credit (the safe direction: temporarily stricter than K) and the GC reconciler repairs the counters from the executing-rows truth each sweep.
• An await releases the slot (parking is not executing); the woken step re-admits through the gate. Prefetch-buffered rows hold slots (they are claimed).

A NULL concurrency_key (the common case) never serializes and pays nothing for any of this machinery.

A saturated gate's backlog crowds its queue. Rows denied by a full gate stay runnable and keep occupying the pick window (unlike K = 1 siblings, which are filtered out for free), so a deep backlog on one saturated key can starve unrelated same-priority work behind it until completions free slots. Give a high-volume gated flow its own queue — see the honest-list entry in the performance notes.

Sharding a big gate

A gate's cap is split across shards slot-counter rows (default 1):

concurrency_limits: [stripe: [limit: 1000, shards: 10]]

Sharding buys two things. Pick-side parallelism: each pick locks the shards it needs with FOR UPDATE OF b SKIP LOCKED, so pickers on different nodes take disjoint shards and admit in parallel instead of serializing (or blocking) on one row — size shards ≥ the nodes that contend the key. Release-side throughput: completions credit back the shard they came from (a row lock held to commit), so one key's completions spread across shard rows instead of serializing at 1 / commit_latency (≈1–3k/s on local disks) — size shards ≥ limit × commit_latency / step_duration (limit 1000, 100 ms steps, 1 ms commits ⇒ 10 shards). Take the larger of the two. A lone picker grabs all shards and admits up to the full cap, identical to an unsharded gate; a gate is only hot if its key is hot, and the cap itself throttles the key, so defaults rarely need touching below limit ≈ 500. shards is clamped to limit (more shards than slots is nonsense).

Releasing or switching the key mid-flight

The key persists across steps by default (identity semantics). A step that no longer needs it can release it, or switch it, per transition:

{:next, "wrap_up", state, concurrency_key: nil}          # release
{:next, "call_api", state, concurrency_key: {:stripe, t}} # switch (admitted at next claim)

⚠ Config names capture key prefixes

concurrency_limits: [order: [limit: 5]] applies to every key named order:* — including ones you meant as mutual-exclusion identities. That silently turns "steps of order:42 never overlap" into "five at a time" and breaks the exclusion. Keep gate names (integrations: stripe, openai) disjoint from identity names (entities: order, account), and treat adding a config as a semantics change for every existing key with that prefix.

Concurrency key vs queue concurrency vs rate limit

They compose: a key can be serialized/gated and rate-limited; the pick admits a row only when it passes both, and debits neither limit for a row the other rejected.