A durable, distributed gen_server backed by a pluggable storage backend.

A dgen_server is an abstract entity composed of durable state and operations on that state. The state lives in the configured backend (default: FoundationDB) and the operations are defined by a callback module implementing the dgen_server behaviour. This allows a gen_server to outlive any single Erlang process, node, or cluster.

Zero or more Erlang processes may act on a dgen_server at any time. Processes with consume enabled consume messages from the durable queue and invoke callbacks; processes without it only publish messages.

Options

The following options may be passed via the Opts proplist:

• tenant (required) - {DbHandle, Dir} pair identifying the backend subspace.

• consume (default true) - whether this process consumes messages from the queue.

• reset (default false) - when true, re-initialise the durable state even if it already exists.

• dead_letter_threshold (default infinity) - number of consecutive processing failures before a message is treated as a dead letter. When a message has failed this many times it is moved to the dead-letter queue, the caller raises {dead_letter, N} (for call messages), and the optional handle_dead_letter/2 callback is invoked. infinity (the default) disables dead-lettering entirely.

• consume_k (default 1) - the maximum number of queued messages a consumer processes per transaction (the batch size). Each consume cycle peeks up to consume_k messages, processes them one at a time while carrying the durable state in memory, and commits the whole batch in a single transaction (the state is read once and written once). It then immediately re-arms itself, so a consumer that has work keeps draining batches.

  Raising consume_k amortises the fixed per-transaction cost (read version, state read/write, commit) across more messages, increasing throughput when the per-message callback work is small. The cost is larger transactions and a coarser unit of progress (a batch is all-or-nothing; on a conflict the whole batch retries). Because a busy consumer keeps draining, a larger consume_k also tends to keep a single node as the active consumer, which is useful when the callback's identity matters — for example an election-style callback where you want to minimise churn in who processes successive messages.

  See "consume_k and inlining" below for how consume_k > 1 changes call handling.

• lock_timeout (default infinity) - maximum number of milliseconds a distributed lock may be held before other consumers treat it as stale. When a callback returns {lock, State}, dgen_server sets a timestamped lock key in the backend that pauses all other consumers while handle_locked/4 runs. Under normal operation the lock holder clears the key itself before handle_locked returns. If the holder is killed (SIGKILL, VM abort) the lock persists until another consumer detects staleness: it re-checks lock_timeout ms after the lock was set and clears it if that deadline has passed. infinity (the default) disables stale-lock detection entirely — a dead holder will permanently block all consumers. Set this to a value safely larger than the worst-case handle_locked duration for your callback.

consume_k and inlining

A message can reach a callback by two routes:

1. Through the durable queue. cast/2 and (normally) call/2 enqueue the message; a consumer later peeks it as part of a consume_k-sized batch and processes it inside the batch's single transaction. This is the ordered, durable path, and the only path when consume_k > 1.
2. Inline. As a latency optimisation, when consume_k =:= 1 a call/2 whose queue is currently empty and unlocked is processed immediately, in the caller's own request transaction, instead of being enqueued and waited on. The result is identical to processing it through the queue; it just skips the enqueue/await round-trip.

When consume_k > 1, inlining is disabled: every call/2 goes through the queue and the batched consume loop, so consume_k is always in effect. Use this when you want all processing funnelled through the batched, single-consumer loop — for example to keep a stable consumer identity (an election-style callback that would otherwise see successive messages committed by different nodes). With the default consume_k =:= 1, inlining is enabled and a contention-free call/2 is served on the fast path.

Independently of consume_k, priority_call/2 and priority_cast/2 always bypass the queue (and locks) and are handled immediately. They are an explicit escape hatch for urgent, usually read-only work; they trade away ordering with respect to queued messages, so prefer call/2/cast/2 for anything that must be ordered with the rest of the stream.

Callbacks

• init/1 - return {ok, State} or {ok, Tuid, State}.
• handle_call/3 - return {reply, Reply, State} or {reply, Reply, State, Actions}.
• handle_cast/2 - return {noreply, State} or {noreply, State, Actions}.
• handle_info/2 - return {noreply, State} or {noreply, State, Actions}.
• handle_dead_letter/2 (optional) - called after a message is dead-lettered. Receives (Msg, AttemptCount). Return value is ignored. Useful for custom alerting or metrics.

Actions is a list of 1-arity funs executed after the transaction commits. The argument is the module state as committed by that transaction — note that under batched consumption (consume_k > 1) all of a batch's actions run after the batch's single commit and each receives the state as of the end of the batch, not the state at the message that produced the action.

Module State

The module that implements the dgen_server behaviour may define any term to serve as the State. dgen_server will encode this state for writing to the database. We encourage you to structure your state to fit in with this encoding scheme, which will yield performance benefits.

• term (fallback) - term_to_binary, chunked into 100KB values.
• assigns map - map with all atom keys; each entry stored at a separate DB key using atom_to_binary(Key) in the path.
• component list - list of maps where every item has an atom id key with a binary value; each item stored separately, ordered by a fractional index embedded in the DB key.

The encoding is applied recursively. For example, an assigns map whose value is a component list will nest both encodings in the key path.