GenDurable.Limiter behaviour (gen_durable v0.2.10)

Copy Markdown View Source

Out-of-band admission for configured limits (rate_limits: / concurrency_limits:).

Admission used to be fused into the pick's single @pick_sql claim (the c_*/r_* CTEs in GenDurable.Queries). Under fan-out that fused claim locks hard, and sharding the buckets (0.2.7) smears the contention without curing it. So admission moves out of the claim: the pick now (1) claims candidate rows into executing, then (2) asks a Limiter backend to admit them, then (3) keeps the admitted and releases the denied. The claim and the admit are no longer one statement — the price is a narrow over/under-admission window, which self-heals exactly as a crash already does (see reconcile/1).

What STAYS in-band and is NOT this module's concern: K=1 mutual exclusion for unconfigured concurrency keys, enforced by the gen_durable_concurrency_active unique index. That is intrinsic to the durable row, not a configured limit.

Backends

A limiter is a {module, handle} pair. module implements this behaviour; handle is whatever connection state it needs (Postgres: %{repo: repo}; Redis: a Redix conn). Call sites use the dispatch helpers (admit/2, credit/2, …), never the backend module directly.

  • GenDurable.Limiter.Postgres — the sharded gen_durable_buckets table, its refill/debit logic, and reconcile_concurrency/gc_buckets, run as standalone statements instead of welded into the pick. Default; no new dependency.

  • GenDurable.Limiter.Redis — Lua. Rate is a token bucket; concurrency is a lease-scored ZSET that self-heals when a holder's lease expires.

Ordering contract

admit/2 receives the batch already ordered by (priority, eligible_at). Rate admission is cumulative per key (grant while the running weight fits the key's available tokens), so order is significant — a backend MUST honour it.

Summary

Types

Admission outcome.

The configured concurrency_key of a gated entry, e.g. "tenant:acme".

One configured limit, pushed to the backend by sync_config/2.

One claimed candidate awaiting admission.

The rate_limit key of a rate-limited entry, e.g. "api:stripe".

Opaque handle a backend returns for each admitted, concurrency-gated entry. It carries whatever the backend needs to credit the slot back on completion (PG: {key, shard}; Redis: {key, member}). Threaded onto the row and handed to credit/2 unchanged.

t()

A backend instance: its module plus the connection state it dispatches on.

Callbacks

Admit a priority-ordered batch. Debits rate tokens and takes concurrency slots for the entries it clears. Debit happens BEFORE the caller flips a row to executing, so a crash in the window leaks in the safe direction (under-admission), healed by reconcile/1.

Credit concurrency slots back — the rows are leaving executing. Rate has no credit (it refills by time). A slot whose backing counter is gone is dropped conservatively.

Self-heal hook the GC calls each sweep. Repairs leaked concurrency slots from the executing-rows truth (available = cap - count(executing)); PG does the reconcile, a lease-native backend (Redis) may no-op since expiry returns slots for free. Returns a stats map merged into the GC's [:gen_durable, :gc, :swept] telemetry.

Extend the leases of slots still held by in-flight/buffered rows — called on the scheduler's heartbeat, alongside the row-lease renewal. A lease-native backend (Redis) bumps each held slot's expiry so a live holder is never pruned mid-step; a backend that counts holders from the executing-rows truth (Postgres) has nothing to renew and no-ops.

Upsert the configured limits. Called on startup and on config change.

Types

admission()

@type admission() :: %{admitted: [{id :: term(), slot()}], denied: [id :: term()]}

Admission outcome.

  • :admitted — entries cleared to execute. Each carries its slot (nil for a rate-only entry, which has no slot to credit).
  • :denied — entry ids to release back to runnable; their limit is biting.

conc_key()

@type conc_key() :: String.t()

The configured concurrency_key of a gated entry, e.g. "tenant:acme".

config()

@type config() ::
  {:rate, name :: String.t(), rate :: number(), burst :: number(),
   shards :: pos_integer()}
  | {:conc, name :: String.t(), capacity :: number(), shards :: pos_integer()}

One configured limit, pushed to the backend by sync_config/2.

entry()

@type entry() :: %{
  id: term(),
  rate: {rate_key(), pos_integer()} | nil,
  conc: conc_key() | nil
}

One claimed candidate awaiting admission.

  • :rate{rate_key, weight} when the step is rate-limited, else nil.
  • :conc — the conc_key when the key is CONFIGURED (has a conc bucket config), else nil. Unconfigured keys are admitted in-band and never reach the Limiter.

rate_key()

@type rate_key() :: String.t()

The rate_limit key of a rate-limited entry, e.g. "api:stripe".

slot()

@type slot() :: term()

Opaque handle a backend returns for each admitted, concurrency-gated entry. It carries whatever the backend needs to credit the slot back on completion (PG: {key, shard}; Redis: {key, member}). Threaded onto the row and handed to credit/2 unchanged.

t()

@type t() :: {module(), handle :: term()}

A backend instance: its module plus the connection state it dispatches on.

Callbacks

admit(handle, entries)

@callback admit(handle :: term(), entries :: [entry()]) :: admission()

Admit a priority-ordered batch. Debits rate tokens and takes concurrency slots for the entries it clears. Debit happens BEFORE the caller flips a row to executing, so a crash in the window leaks in the safe direction (under-admission), healed by reconcile/1.

credit(handle, slots)

@callback credit(handle :: term(), slots :: [slot()]) :: :ok

Credit concurrency slots back — the rows are leaving executing. Rate has no credit (it refills by time). A slot whose backing counter is gone is dropped conservatively.

reconcile(handle)

@callback reconcile(handle :: term()) :: %{optional(atom()) => non_neg_integer()}

Self-heal hook the GC calls each sweep. Repairs leaked concurrency slots from the executing-rows truth (available = cap - count(executing)); PG does the reconcile, a lease-native backend (Redis) may no-op since expiry returns slots for free. Returns a stats map merged into the GC's [:gen_durable, :gc, :swept] telemetry.

renew(handle, slots)

@callback renew(handle :: term(), slots :: [slot()]) :: :ok

Extend the leases of slots still held by in-flight/buffered rows — called on the scheduler's heartbeat, alongside the row-lease renewal. A lease-native backend (Redis) bumps each held slot's expiry so a live holder is never pruned mid-step; a backend that counts holders from the executing-rows truth (Postgres) has nothing to renew and no-ops.

sync_config(handle, configs)

@callback sync_config(handle :: term(), configs :: [config()]) :: :ok

Upsert the configured limits. Called on startup and on config change.

Functions

admit(arg1, entries)

@spec admit(t(), [entry()]) :: admission()

See admit/2.

credit(arg1, slots)

@spec credit(t(), [slot()]) :: :ok

See credit/2.

reconcile(arg)

@spec reconcile(t()) :: %{optional(atom()) => non_neg_integer()}

See reconcile/1.

renew(arg1, slots)

@spec renew(t(), [slot()]) :: :ok

See renew/2.

sync_config(arg, configs)

@spec sync_config(t(), [config()]) :: :ok

See sync_config/2.