The reactive propagation loop — the heart of the substrate, shared by both hosts.
Given a compiled Plan and a host config (recompute strategy + key_rule),
drain the frontier to empty:
- Pick the dirty cell with the smallest depth (
Frontier.next_cell) — no cell recomputes while an input is still dirty (topological order, no external scheduler). - Atomically claim its dirty keys (
Frontier.claim— delete-returning). - Recompute via the host's
RecomputeStrategy→ the keys that changed. - Propagate: mark the changed keys on the cell's parents, applying the
host's
KeyRule(Graph.dirty_parents). - Repeat until empty.
A leaf carries no recompute — a source writes its tuples and marks parents dirty directly, so a leaf shouldn't appear in the frontier; if one does, its claimed keys are treated as changed and just propagated.
Returns {:ok, %ReactiveDag.Drain.Report{}} — the processing trace: one
step per cell recompute (cell, pass, claimed, changed, triggered_by,
duration_us), in execution order, plus run totals. The drain knows all of
this as it works; the report is that knowledge kept instead of discarded.
Persistence is the host's (an Oban job's meta, a run table) — the library
reports, the host records.
Concurrency
The per-cell claim is atomic (a DELETE … RETURNING): a dirty KEY is
consumed exactly once. But the pick-then-claim PAIR is not serialized — two
concurrent drains over the same graph can select the same cell and both
recompute it (each claiming a disjoint slice of its keys). Run ONE drain at a
time per graph (both hosts do: a single worker), or make recomputes
idempotent so a doubled recompute is merely wasted work.
run/2 opts:
:recompute— aReactiveDag.RecomputeStrategymodule (required unless the graph is leaves-only).:key_rule— aReactiveDag.KeyRulemodule (default: identity mapping).:max_passes— runaway guard (default 100000): exceeding it raisesReactiveDag.Drain.RunawayError, whose:reportfield carries the partial trace — the step list showing which cells keep re-dirtying each other is exactly the diagnostic for the cycle the guard suspects.
Telemetry
The drain emits :telemetry events, so a host observes it without threading a
callback through every call site — a dashboard, a metrics backend and a log
can all attach independently, and none of them changes how the drain is
invoked.
| event | measurements | metadata |
|---|---|---|
[:reactive_dag, :drain, :start] | system_time | cells (count in the plan) |
[:reactive_dag, :drain, :step] | duration_us, claimed, changed | cell, pass, changed_keys, triggered_by, step |
[:reactive_dag, :drain, :stop] | duration_us, passes, steps, changed | report, cells_touched |
[:reactive_dag, :drain, :exception] | duration_us | kind, reason, report |
:step carries the changed KEYS, not just their count, because that is what
makes a consumer incremental: a dashboard that knows which cells moved reads
only those instead of re-reading the graph.
:telemetry.attach("my-drain-log", [:reactive_dag, :drain, :stop], fn _e, m, meta, _ ->
Logger.info("drained #{length(meta.cells_touched)} cells in #{m.duration_us}us")
end, nil):exception fires for a RunawayError too, carrying the partial report — a
monitor should see the runaway, not just the crash.
This replaces an earlier :on_step option. A closure threaded through run/2
could only serve whoever owned that call site — a dashboard, a metrics backend
and a log could not all have one, and adding a second consumer meant editing
every place the drain was invoked. Telemetry has no such limit, and a caller
who wants a plain closure can attach one in three lines.
Summary
Functions
@spec run( ReactiveDag.Plan.t(), keyword() ) :: {:ok, ReactiveDag.Drain.Report.t()}