Telemetry and the OpenTelemetry bridge half

Copy Markdown View Source

This note is the design record for what statifier_persistence emits and what it deliberately leaves to others. It was written as a specification ahead of the code; the family-two emit sites landed in sp-m0i and the family-one ones in sp-t01, so both tables below now describe what this package emits rather than what it promises to. Nothing in the contract changed on the way in - docs/adr/0009-telemetry-events-for-the-durable-stepper.md froze it, and decision 8's amendment discipline is how it moves.

One row is specification still, and the table says so where it sits: [:statifier, :session, :unroutable] has no emit site, because no seam in this package can currently produce the case it names.

Four records govern and are not restated here:

  • st-ADR-0062 and statifier-ex's docs/opentelemetry.md - the family's span topology and the ruling that the OpenTelemetry bridge is one separate package, opentelemetry_statifier, consuming public :telemetry events only. Its "What lands where" table gives this repository two rows, and both are this note's mandate.
  • st-ADR-0067 - one telemetry contract across stepping drivers. Its decision 2 puts the emitters in Statifier.Telemetry; decision 3 tabulates what a process-less driver emits; decision 4 adds the driver metadata key; decision 6 says the storage phases are this package's own surface, under [:statifier_persistence, ...].
  • st-ADR-0040 (as amended by st-ADR-0067) and Statifier.Telemetry - the family's event conventions, adopted here in full except where this note records a deliberate departure.
  • sob-ADR-0006 and statifier_oban's docs/telemetry.md - the sibling that went first, whose precedents this note adopts and, twice, deliberately departs from.

Two families, and why this package has two

Every other sibling in the family emits one event family of its own. This package emits that, and it also emits the interpreter's family, because it is a stepping driver.

Family one: [:statifier, :session, ...], emitted through Statifier.Telemetry with driver: :persistence. An execution stepped here produces the same 27-name contract an execution hosted in a Statifier.Session does - the same macrostep spans, the same effect events, the same counters - because this package calls the same functions session.ex calls rather than reimplementing a documented table. This is the whole point of st-ADR-0067: a backend user should not have to know that a durably-stepped macrostep is a different kind of thing, because it is not. The one difference is the statifier.driver attribute.

Family two: [:statifier_persistence, ...], this package's own. Everything about how a position got into memory and back out: the serialized unit, the lock, the adapter calls, the identity guard, the execution record's lifecycle, the executor seam's failures, and the durable-subchart seam. st-ADR-0067 decision 6 draws that line and names this namespace.

The two nest. A durable macrostep span appears inside the step span this package opens around it, because the bridge parents it from its own pid-keyed span table (ots-ADR-0004 decision 4) rather than from the process's ambient OTel context, which it never reads; both are emitted in the same process during the same synchronous call, which is what puts them in one row's reach.

Family one: what this package emits as a driver

Applicability is st-ADR-0067 decision 3's table. What that means here, concretely, at the emit sites:

EventEmitted by this package?Where
[:statifier, :session, :init]yes, exactly once per logical executionExecutions.create/4, around Interpreter.initialize/2, resumed: false, invoked_by: nil. Never on a load - every Executions.step/5 is a rehydration, and an :init per load would fire thousands of times per execution
[..., :halt]yesthe step whose outcome is terminal, once its write has landed. reason is :done or :budget_exhausted; fail/4 and cancel/3 reach no interpreter and emit none
[..., :terminate]neverit names a GenServer callback; there is no process here, and both halves of every span arrive inside one call, so there is no open-span entry to leak
[..., :macrostep, :start] / [..., :stop]yesbrackets each Interpreter advance call: initialize/2 in Executions.create/4 (trigger: :initialize), handle_event/2 in Executions.stepped/7 (trigger: :event), and each deliver_internal/5 re-entry wave (trigger: :internal, nested, per st-ADR-0067 decision 5)
[..., :interpret]neverthis package has no st-ADR-0029 injection seam. If it grows one it emits this event rather than minting a name
[..., :unroutable]contract only - no emit site todayit names an effect nothing could route: no dispatch arm, and no executor accepted its kind. The executor seam's only verdicts are :ok and {:error, reason} (StatifierPersistence.Executor), and Driver's own dispatch ends in an accepting catch-all, so the case is absent by circumstance rather than inapplicable. An executor that accepted the effect and returned {:error, reason} is not it; that is [:statifier_persistence, :effect, :failed]. A seam that can refuse an effect outright emits this event rather than minting a name
[..., :effect, _] (11)yesevery effect the advance produced, in the core's own list order, from persist_tail/7 - lifecycle effects (:done, :budget_exhausted) included, since the bridge needs them even though the executor never sees them
[..., :trace, _] (9)yes, under trace: truethe same pass; the flag rides the position (st-ADR-0060) and the gate stays in the core, which simply produces no trace effects when it is off

driver: :persistence is on every one of them, and it is frozen (ADR-0009 decision 2). st-ADR-0067 open question 1 left the atom to this repository; this is the answer, and changing it later is a breaking change to a real consumer.

The session_id these emitters take is the chart's own _sessionid, read out of the decoded datamodel - the same read Driver already performs for event origin. This package never performs an extra lookup to obtain it, and never invents one.

st-ADR-0067 decision 4's rule holds unchanged: no execution_id on this family. The storage key is this package's vocabulary and travels on family two.

Two shapes a reader of the emit sites will notice, both deliberate:

  • The effect events are emitted up front, not interleaved with execution. They report what the chart produced; what the host's executor then made of each one is family two's [:statifier_persistence, :effect, :failed]. Interleaving would also have to place the two lifecycle effects the executor never sees somewhere other than where the interpreter put them.
  • The :initialize span is the one span not nested inside a step span. Interpreter.initialize/2 runs in Executions.create/4 before the per-execution exclusion opens, so the bridge has no step span recorded for that process when it fires: it is not a child of the create's [:statifier_persistence, :execution, :step, :start]/:stop pair, and with nothing of the bridge's own open around it, it is the root of its own trace. Every other macrostep span is emitted inside the serialized unit and nests as st-ADR-0067 decision 6 expects. Both halves of every one of them are emitted inside one synchronous call, so decision 5's "a span never crosses a persist boundary" holds structurally.

Family two: the storage-phase contract

Conventions adopted from the family

  • Measurements are numbers; metadata is everything else, integer-valued indexes included, because an opaque index has no numeric meaning to average. child_index is therefore metadata.
  • One definition site, enumerable ahead of a call. StatifierPersistence.Telemetry builds every name from module attributes holding literal atoms, and events/0 returns the full list, the way Statifier.Telemetry.events/0 and StatifierOban.Telemetry.events/0 do. The bridge attaches one handler per event name under its own handler id and cannot do that for a list it has to hand-copy. Deriving a name segment from a module name at runtime is forbidden for the same reason (and by Credo's UnsafeToAtom).
  • Moduledoc shape: a one-line opener naming ADR-0009, a section per structural rule, then a markdown table of Event | Measurements | Metadata per family, with each family's emission gate stated above its table. Each emitter's @doc names the exact event it emits.
  • Amendment discipline. Adding a measurement or metadata key to an existing event is an amendment and is fine; renaming or removing one, renaming an event, or changing the driver atom is breaking and needs a new ADR. A bridge that needs data these events lack gets a new field here - it never reaches into this package.
  • No configuration knob and no sampling knob. Emission is unconditional: :telemetry.execute/3 on an event with no handlers is a lookup and a return, and every option this package carries is a seam a host must state explicitly (ADR-0002). Nothing in this family scales with microstep count, so upstream's trace: true gate has no counterpart.

Deliberate departures from sob-ADR-0006, and why

statifier_oban landed its half first and invited a sibling with a genuinely different shape to say so rather than copy out of deference. Two of its decisions do not transfer, and one does.

Adopted: the [:package_name, ...] prefix, fixed and not configurable, because the bridge must name the events at compile time and a per-host prefix would make its attach list depend on host configuration it cannot see. st-ADR-0067 decision 6 names this exact namespace independently.

Departure 1: the identity key is execution_id, not scope. StatifierOban.Timer.Key's scope is either a live session's id or a host's durable execution id and that package cannot tell which, so scope is the only honest name available to it. Here both exist as distinct named things: execution_id is this package's storage key, present on every seam, and the logical _sessionid is upstream's correlation key carried in the position blob. Naming either of them scope would discard information this package has. session_id rides where a position has already been decoded and is explicitly nil otherwise.

Departure 2: the step seam is a :start / :stop pair. That record's rule - no pairs - rests on Oban already owning every interval it could bracket. This package owns an interval nobody else measures: lock, load, decode, identity-check, advance, execute effects, persist. The upstream macrostep span is expected to nest inside it, and the bridge's nesting needs an outer span that is genuinely open to record in its table, which a single event carrying a duration cannot provide. span_ref keeps st-ADR-0040 decision 2's semantics exactly: a fresh make_ref/0 per span, on both halves, the only pairing key. Both halves are emitted inside one function call, so st-ADR-0067 decision 5's "a span never crosses a persist boundary" holds structurally.

Everything that is not the step seam is a single point-in-time event, on that record's own reasoning - except the batch migration, the second span (below): one call over every execution on a chart hash is an interval this package owns too.

The step seam

Brackets one serialized drive - Executions.create/4, Executions.step/5, Executions.fail/4 or Executions.cancel/3 inside serialized/5. Emitted on the calling process. The [:statifier, :session, :macrostep, ...] span opens and closes inside it.

EventEmitted fromMeasurementsMetadata
[:statifier_persistence, :execution, :step, :start]Executions, immediately inside serialized/5system_time, monotonic_timeexecution_id, entry, span_ref
[:statifier_persistence, :execution, :step, :stop]the same call, on every return pathduration, monotonic_timeexecution_id, session_id, content_hash, entry, outcome, status, reason, span_ref, invoke_id, child_count, selection
[:statifier_persistence, :execution, :step, :exception]the same call, in place of the stop, when the drive raises, throws or exitsduration, monotonic_timeexecution_id, entry, span_ref, kind, reason, stacktrace
[:statifier_persistence, :execution, :step, :reentered]Executions, once per error.communication re-entry the persist tail delivered, between the start and the stopsystem_timeexecution_id, session_id, content_hash, name, origin, opts
[:statifier_persistence, :execution, :lock]serialized/5, after strategy.with_execution/3 returns or refuses; Executions.unpark/3, the same way, outside any step spanduration (the wait, not the held time), system_timeexecution_id, strategy, outcome, reason

entry is which public door was used: :create, :step, :done_invocation, :failed_invocation, :answer_parent, :fail, :cancel. It is a fixed vocabulary and it is the dimension an operator slices by first, because a :done_invocation step and a :step step have different expected shapes.

outcome on the stop is :ok, :discarded or :error, mirroring the three return shapes exactly. status is the execution's resulting Adapter.execution_status (:active, :completed, :failed, :cancelled) and is nil when the step did not reach a write. A step never results in the fifth arm, :needs_migration (ADR-0014): a delivery to a parked execution reaches no write, so its stop carries status: nil, outcome: :error and the bare atom :needs_migration as reason rather than the error term, which carries the execution. reason is the error term on an :error outcome and nil otherwise - and it is a term, so a consumer folding it into a metric dimension must narrow it first.

session_id is nil on the stop when the step never got as far as a decoded position: a terminal-execution discard reads the execution record only, and a lock refusal or an identity refusal never loads at all.

selection on the stop tells the caller whether the event the step delivered selected a transition, without trace: true: :selected when it selected at least one, :none when it selected none. It is read off the returned position's last_selection (Statifier.MachineState.last_selection/0, statifier 2.9), which the interpreter writes on every external round whether or not tracing is on. It is set on every :ok stop whose step delivered an event - entry :step, :done_invocation, :failed_invocation or :answer_parent - and it is the delivered event's own answer: the eventless and internal rounds that follow in the same macrostep, and an error.communication the persist tail re-enters, do not change it. It is nil on a :create, :fail or :cancel stop, where no event is delivered, and on every :discarded or :error stop, which returns no position. :none does not say why nothing was selected: an event no transition names and one whose every matching guard was false read the same. step/5's return value is unchanged; a caller holding the returned position can read last_selection from it directly.

A step span closes exactly once: with :stop on a return, or with :exception when anything inside the drive raises, throws or exits - the host's executor or event builder, an adapter, the serialization strategy. The raise then reaches the caller unchanged, with its original stacktrace; nothing is rescued to a return value. The keys are the ones :telemetry.span/3 puts on its own :exception event: the start half's execution_id, entry and span_ref, plus kind (:error, :throw or :exit), reason and stacktrace. The raised term and its stacktrace can carry any value the failing code held - an event builder is called with the decoded machine state, and a failed match on it raises with the whole state, datamodel included - and the datamodel is never on an event (see "Cardinality and disclosure" below). So the two fields that could carry state are narrowed before the event is emitted, and every other field is the start half's own:

  • reason is the exception's module for an :error, a raw Erlang error normalized first (a failed match reports MatchError); for a :throw or an :exit it is the thrown or exit atom, or :redacted for any other term. The raised value itself never travels.
  • stacktrace keeps each frame's module and function, replaces an argument list by its arity, and keeps only :file and :line of the location, dropping anything else a location carries.

The caller's re-raise is untouched: it sees the original reason and stacktrace. A bridge that closes a span on :telemetry.span/3's exception can close this one the same way.

[:statifier_persistence, :execution, :step, :reentered] is for a host that event-sources an execution: it keeps the events it delivered and folds them to rebuild the position. An executor failure on an actionable effect re-enters the chart as error.communication inside the persist tail (ADR-0004 decision 4), and that event is one the host never delivered, so a fold over the host's own events alone diverges from the persisted position on that edge. This event exposes each such re-entry, once it has been delivered, with the three values it was delivered with: name ("error.communication"), origin (the Statifier.Event.Cause.origin/0 tuple) and opts ([sendid: id] for a failed <send>, [] otherwise). It is emitted on the calling process after the step's start and before its stop, in delivery order, so a handler that appends each one after the event that drove the step, and folds them through Statifier.Interpreter.deliver_internal(machine_state, :platform, name, origin, opts), reaches the persisted position. A failure whose re-entry was not delivered - an observational effect, a failure after the execution reached a final state, a failure after the macrostep budget ran out - emits none. The event carries no span_ref: it pairs with its step by execution_id and by arriving between that step's start and stop. step/5's return value is unchanged.

invoke_id and child_count are the settlement dimensions, and they are nil on every ordinary drive. StatifierPersistence.Driver sets them beside entry: :answer_parent, so the step that carries a fan-out's whole assembled answer through the parent's door is recognisable as that one and not as any other invocation answer; child_count is nil for a single-child subchart, which has no width. They are metadata rather than measurements because they are dimensions of the span rather than quantities it measured - the departure from the numbers-are-measurements split that the ADR-0009 sp-8wv amendment records.

[:statifier_persistence, :execution, :lock]'s duration is the wait for the per-execution exclusion, which is the number that says whether a host's concurrency is fighting itself, and it is invisible from every other surface. strategy is the serialization module (StatifierPersistence.Serialization.AdapterLock by default); outcome is :acquired or :unavailable; reason on :unavailable is the {:serialization, term} payload, including {:serialization, :not_supported} for an adapter that exports no lock_execution/3. Executions.unpark/3 takes the same exclusion and emits the same event, with nothing around it: an unpark is not a step and opens no step span (ADR-0014's telemetry amendment).

The storage seam

Emitted from the Storage facade, above every adapter, on the calling process.

EventEmitted fromMeasurementsMetadata
[:statifier_persistence, :adapter, :call]every Storage facade function, around the adapter callduration, system_timeadapter, callback, outcome, reason, execution_id, session_id, content_hash
[:statifier_persistence, :identity, :refused]Storage's own precheck_identity/4, every writer's identity arm, and persist_tail/7system_timeexecution_id, session_id, stage, reason, stored_content_hash, supplied_content_hash

callback is the name of the Storage.Adapter callback the facade called - :init, :save_chart, :fetch_chart, :save_position, :fetch_position, :insert_execution, :fetch_execution, :update_execution, :supports_metadata?, :list_executions_by_metadata, :supports_execution_outcome?, :list_execution_states_by_metadata, :supports_content_hash_query?, :count_executions_by_content_hash, :list_active_execution_ids_by_content_hash, :list_execution_ids_by_content_hash, :supports_chart_retirement?, :retire_chart, :supports_retired_info?, :fetch_retired_info, :supports_input_log?, :append_input, :list_inputs, :supports_execution_pruning?, :prune_executions - a closed vocabulary fixed by the behaviour. The behaviour's other two callbacks never appear: isolate/1 is called by the conformance suite alone, and lock_execution/3 is taken through the serialization strategy, which [:statifier_persistence, :execution, :lock] reports. A retired-chart check on an adapter that declares supports_retired_info?/1 reports :supports_retired_info? and :fetch_retired_info where one on any other adapter reports :fetch_chart, so a handler that matches callback exhaustively names all three. adapter is the module. Between them they answer "which storage call is slow" without a host instrumenting its own adapter, and they answer it for the in-memory adapter too, which no SQL tracer sees.

outcome is :ok or :error; reason carries the adapter's own error arm (:chart_not_found, :position_not_found, :execution_exists, :execution_not_found, :metadata_unsupported, {:adapter, term}) and the facade's capability refusals (:child_listing_unsupported, :execution_position_missing, :not_a_statifier_blob, {:unsupported_format_version, term}). Which of execution_id, session_id and content_hash are present is whatever that callback is keyed by; the rest are nil.

[:statifier_persistence, :identity, :refused] is the sharpest event in this contract. stage is :position, :execution or :chart; reason is :identity_mismatch or :unidentified_chart. A mismatch means a chart revision changed under a live execution - the exact drift the guard exists to catch (ADR-0003) - and a host that cannot count it learns about the deploy that caused it from a support ticket. Only the two content_hash values travel, never the Identity structs the error term carries; see "Cardinality and disclosure" below.

An adapter call inside a lock inside a step nests three deep in the bridge, through its own span table, with no propagation machinery involved.

The execution lifecycle seam

EventEmitted fromMeasurementsMetadata
[:statifier_persistence, :execution, :created]Executions.create/4, after the insertsystem_timeexecution_id, session_id, content_hash, child?, metadata?
[:statifier_persistence, :execution, :terminated]Executions.create/4, Executions.step/5, Executions.fail/4, Executions.cancel/3, on any terminal writesystem_timeexecution_id, session_id, content_hash, status, driven_by, reason
[:statifier_persistence, :execution, :discarded]Executions.step_tail/7, step_loaded/8, repair_terminal/4, and fail/cancel's terminal armssystem_timeexecution_id, entry, reason, repaired?
[:statifier_persistence, :execution, :migrated]Executions.migrate/4, after its serialization section returns; Executions.migrate_tree/4, once per node it re-pins, children first, after every exclusion is releasedsystem_timeexecution_id, from_content_hash, to_content_hash, dropped
[:statifier_persistence, :execution, :unparked]Executions.unpark/3, after its serialization section returns, when it wrote :activesystem_timeexecution_id, content_hash
[:statifier_persistence, :effect, :failed]execute_effects/3 and reenter_failures/4system_timeexecution_id, session_id, content_hash, kind, executor, reason, reentered?
[:statifier_persistence, :drive, :turns_exhausted]Driver's turn loop, on {:turns_exhausted, n}system_time, turnsexecution_id, entry

driven_by on :terminated is :chart or :host, and it is the reason this event exists at all. A chart-driven termination is also a [:statifier, :session, :halt]; Executions.fail/4 and Executions.cancel/3 are not. No interpreter runs on those paths - ADR-0004 decision 6 makes abandonment a host decision about the execution rather than a chart transition - so upstream emits nothing, and a host counting :halt alone would undercount its own terminations by exactly the ones it caused itself. status is :completed, :failed or :cancelled; reason is the execution record's short failure string, which ADR-0004 decision 1 already constrains to be console-readable, or nil.

:discarded's reason is a closed vocabulary of the three ways a delivery becomes a non-event: :terminal_execution (the record was already terminal, read before any position decode), :builder_declined (an event builder returned :discard under the exclusion), and :position_terminal (Interpreter.handle_event/2 returned {:error, :not_running}, the structural backstop for a record whose :active status lied). Only the third sets repaired?: true, and a host seeing it regularly has a durability bug upstream of this package - which is precisely why the repair path is worth a countable event rather than a silent fix.

[:statifier_persistence, :execution, :migrated] is the one event a migration emits, once per execution Executions.migrate/4 re-pins, and nothing is stored as a trace of it (ADR-0013 decision 5). Executions.migrate_tree/4 is its second emitter and adds no key: one event per node its one store unit re-pinned, the children's before the root's, emitted once the unit has returned and every exclusion is released (ADR-0015 decision 5). A migration has two charts in hand, so the event names both hashes rather than carrying one content_hash: from_content_hash is the chart the execution was pinned to and to_content_hash the chart it is pinned to now. dropped is the list of state ids the plan dropped that were in the execution's configuration, [] when it dropped none; a drop is an operator exit and is never an authored transition, so it has no family-one event of its own. A migration is not a step: it opens no step span and takes no entry. A refused or parked migration emits nothing from this family, and whether it should is left open by ADR-0013. A refused or parked tree emits nothing either, for any node.

[:statifier_persistence, :execution, :unparked] is an unpark's own event, beside the lock event and its adapter calls: once per Executions.unpark/3 that writes a :needs_migration execution back to :active, after the serialization section returns. content_hash is the chart the execution was parked on and goes on under, the one chart an unpark has in hand. session_id is not on it, because an unpark decodes no position. An unpark that writes nothing emits no :unparked: an :active execution, which answers {:ok, execution} unchanged, a terminal one, which is discarded with no :discarded event, an absent one, and a refused lock. An unpark is not a step: it opens no step span and takes no entry (ADR-0014's telemetry amendment).

[:statifier_persistence, :effect, :failed] is where the executor seam's verdicts land. kind is the effect's kind atom, executor is the module (or :fun for the arity-2 form), reason is the {:error, reason} term it returned, and reentered? says whether ADR-0004 decision 4's error.communication re-entry was opened for it - false for an observational effect, whose failure is discarded because observation must never steer an execution, and false for a failure inside a re-entry wave, which is single-wave by design. Nothing wraps a successful executor call: the host's work is the host's to instrument, and the step span already bounds it.

The batch migration span (ADR-0017 decision 6)

Brackets one Executions.migrate_batch/3 call, opened once its options are checked. Emitted on the calling process.

EventEmitted fromMeasurementsMetadata
[:statifier_persistence, :execution, :migrate_batch, :start]Executions.migrate_batch/3, after its options are checked and before the plan's checks and the listingsystem_time, monotonic_timefrom, to, dry_run, span_ref
[:statifier_persistence, :execution, :migrate_batch, :stop]the same call, on every returnduration, monotonic_time, one count per outcome of the modefrom, to, dry_run, span_ref, outcome, reason
[:statifier_persistence, :execution, :migrate_batch, :exception]the same call, in place of the stop, when the batch raises, throws or exitsduration, monotonic_timefrom, to, dry_run, span_ref, kind, reason, stacktrace

from and to are the plan's two content hashes, and dry_run is the option the call was given. The stop's counts are the report's counts, one measurement per outcome of the mode, every one present: would_migrate, would_refuse and skipped under the dry run; migrated, refused, parked and skipped under the apply. outcome is :ok when the call answered a report, with reason nil, and :error when it refused the whole batch, with reason the refusal it answered and every count 0.

The dry run opens the span and so does a whole-batch refusal, so a host counting stops counts every batch it asked for. A malformed option raises ArgumentError before the span opens. The exception's keys are the step span's: the start's metadata plus kind, reason and stacktrace, with reason and stacktrace narrowed the same way, and the raise then reaches the caller unchanged.

Each execution the apply moves still emits its own [:statifier_persistence, :execution, :migrated], unchanged, between the batch's start and its stop; the dry run emits none. A batch is not a step: its span takes no entry and no execution_id.

The durable-subchart seam (ADR-0008)

Emitted on the parent's stepping process, at dispatch time.

EventEmitted fromMeasurementsMetadata
[:statifier_persistence, :child, :started]Driver.start_child/3, after adopt_child/3system_timeparent_execution_id, child_execution_id, invoke_id, child_index, content_hash, session_id
[:statifier_persistence, :child, :refused]the same chain, on any refusalsystem_timeparent_execution_id, invoke_id, reason
[:statifier_persistence, :child, :recorded]Driver.record_and_settle/5, after the child's own answer is writtensystem_timeparent_execution_id, child_execution_id, invoke_id, child_index, outcome
[:statifier_persistence, :child, :answered]Driver.answer_parent/3, after the parent's door returns; Driver.resolve_and_answer_parent/3, when the parent is never reachedsystem_timechild_execution_id, parent_execution_id, invoke_id, outcome, child_count, failed_count, delivery
[:statifier_persistence, :child, :settled]Driver.settle/3, once per decisionsystem_time, child_count, completed, failed, cancelled, unstartedparent_execution_id, invoke_id, policy, decision
[:statifier_persistence, :child, :cascade_cancelled]Executions.cascade_cancel/3, after the sweepsystem_time, count, retainedparent_execution_id, invoke_id

content_hash on :started is the child's pinned hash - ADR-0008 decision 2's mandatory identity pin, recorded in the child's linkage metadata - which is what lets a consumer tell a child restarted against a redeployed chart from one that was not. session_id is the child's own logical session, so the bridge can stitch the child's macrostep spans to this event without reading Execution.Linkage.

:refused's reason is ADR-0008 decision 4's closed refusal set: :child_listing_unsupported (the adapter cannot host children), :unidentified_chart (the resolved child chart carries no identity), :execution_exists, and a Statifier.Invoke.Source.resolve/2 reason. All four reach the chart as {:failed, reason: "child_execution_creation_failed", detail: detail}, so a host sees them there too - but only as a chart-level failure, without which of the four it was.

:answered's outcome is :done or :failed. For a single child it mirrors the two doors, and child_count and failed_count are nil: one child is not an invocation with a width. For a fan-out it is the invocation's aggregate - :failed when any index failed - which is not the door: a fan-out that lost an index still answers through done_invocation/5 with a dense list, because st-ADR-0068's failure shape is inside the entry rather than around the list. Reporting the door said outcome: :done for a settlement that failed, which is the one thing a consumer counts this event to learn (the ADR-0009 sp-8wv amendment). failed_count is how many entries in that list failed.

:answered's delivery is what the parent's door answered, from a closed vocabulary: :delivered (the parent took the answer), :discarded (the parent had already left the invocation, ADR-0007 decision 3), :needs_migration (the parent is parked and refused the answer whole, ADR-0014 decision 2) or :error (any other error, whose term does not travel). The automatic answer - the one a child's own drive makes once it is terminal - returns the child's own result whatever the parent's door answered, so this key is where a refused answer reaches the host. The event is emitted on the process driving the child, before that drive returns, so a handler seeing delivery: :needs_migration runs beside the caller holding the child's result; delivering the answer again through Driver.answer_parent/3 once the parent leaves the arm is the host's. For a fan-out the children's recorded answers stay on their own executions, and answering again through any one child settles the invocation again (the ADR-0009 sp-6neq amendment). A single child's answer is recorded on its own execution too, before the parent's door is tried, so a host that no longer holds it reads it back: StatifierPersistence.Execution.from_record/1 over the child's fetched record carries it as donedata, and a failed child's failure was always on the record (ADR-0008's 2026-09-24 Amendment).

Two more delivery values say the answer never reached a door at all. The automatic answer, and Driver.resolve_and_answer_parent/3 with a chart_resolver:, first fetch the parent's own record and then resolve its chart through the resolver; :parent_unfetched is a parent whose record did not fetch, and :parent_chart_unresolved is one whose chart the resolver did not return. Neither the fetch's error nor the resolver's answer travels. No door ran and no settlement was entered, so on these two outcome is the child's own (:done or :failed), child_count is the linkage's, and failed_count is nil even for a fan-out. The child's own drive still returns its own result, and resolve_and_answer_parent/3 still returns :ok; delivering the answer once the parent can be reached is the host's, through Driver.answer_parent/3 with a driver over the parent's chart (the ADR-0009 sp-q0pp amendment).

:recorded and :settled are the fan-out settlement seam, and both are emitted inside the parent's settlement exclusion, which is what makes them trustworthy: a :recorded cannot claim an answer the settlement that follows will not read.

:recorded fires once per child answer written to a child's own execution record. Every index but the settling one records an answer that reaches no door at all - nine of ten for a ten-wide fan-out - so this is the only surface those answers appear on.

:settled fires once per decision the settlement section reaches, and not at all for a read that failed before reaching one. decision is :answer or :not_yet, and it is the settlement's decision, not the parent's answer to it: :answer says every index is settled and the assembled list goes to the parent's door, and a parked parent that then refuses the list still reports :answer here. What the door answered is the delivery of the :answered event that follows (ADR-0008's 2026-09-24 Amendment). The four tallies are read off the same states the decision was made from, the first_error cancels included, and unstarted is the indexes with no execution of their own at all - which is what tells a fan-out still starting from one that is stuck. They partition child_count only once every index has an execution.

count on :cascade_cancelled is how many executions the sweep actually cancelled and retained is how many it found already terminal and left alone - ADR-0008 decision 5's retain semantics as a number. Both are legitimately 0: a cancel matching nothing is a no-op, not an error, and a crash-recovering host may replay a cancel whose start it never durably recorded. invoke_id is nil for the whole-parent sweep and set for the per-invocation one.

A child is a separate execution with its own logical session, so its own steps produce their own step spans and their own macrostep spans - not children of the parent's. Parenthood would hold the parent's trace open for the child's whole life, which on this package's target hosts is days. The bridge links instead, from :started.

Cardinality and disclosure

Every metadata key above is bounded by the chart or by a closed vocabulary, with three exceptions, all deliberate.

execution_id is host-supplied and unbounded. It is present as a correlation id for a span or a log line, never as a metric dimension - the same status job_id has in statifier_oban and id has in Oban itself.

opts on [:statifier_persistence, :execution, :step, :reentered] carries a <send>'s sendid, which is the element's own id or one the interpreter generated for it, so it is unbounded like execution_id: a value for a fold to replay, never a metric dimension. origin beside it is indexes into the chart and is bounded by it.

reason is a term on several events. Where this note names a closed vocabulary (:discarded, :child, :refused, the adapter arms) it is safe to dimension on. Where it carries an arbitrary executor or adapter error (:effect, :failed, :adapter, :call's {:adapter, term}, the step stop, the batch migration stop), a consumer must narrow it before it becomes a dimension. The step exception's reason is already narrowed to a module or an atom. A host executor returning a per-effect struct there will blow up any metric keyed on it, and no change here can prevent that.

Nothing host-opaque and nothing from the datamodel is ever on an event. Never emitted, in any form - not truncated, not hashed, not "just the keys": the chart_blob, the position_blob, the identity_blob, the ADR-0006 metadata map, the chart's datamodel, an invoke's params, and a :done effect's donedata.

The metadata map is the one a well-meaning implementer will reach for, because it holds exactly the tenant and correlation ids an operator wants to slice by. It is excluded anyway. It is unbounded by construction and host-defined, so no cardinality budget can be reasoned about here; and ADR-0006 decision 2 already records that it sits at rest in the clear outside :blob_type encryption, so a host that filed something it should not have would have it leave the database on this channel. metadata? on [:statifier_persistence, :execution, :created] is a boolean - whether a non-empty map was supplied - and that is the whole of what this contract says about it. A host wanting its own dimensions has execution_id on every event and its own table to join.

The identity guard's refusal carries the same rule into a place it is easy to miss. The error term is {:identity_mismatch, stored, supplied} with two whole Identity structs; the event carries the two content_hash values and nothing else from them. A content hash is a digest of a chart document and is the key this package and its host already exchange in the open; the envelope around it is not.

caller_context does not appear in this family at all. This package originates none. It rides family one from the driving event, where Statifier.Telemetry puts it and where st-ADR-0063 intends it, and the bridge reads it there.

The bridge half

opentelemetry_statifier is the only package in the family that calls an OpenTelemetry API (st-ADR-0062), and it bridges siblings as separate per-library setup calls. This package's half of that bridge is an obligation, not code: the promise that the events above carry everything the bridge needs, so it never reads a chart blob, a position blob, an execution record, StatifierPersistence.Execution.Linkage, or an ADR-0006 metadata map. Span construction, handler attachment and the span table are the bridge repo's decisions and are not specified here.

What the events are built to let the bridge do:

  • Family one needs no bridge work at all. These are upstream's own 27 events with driver: :persistence in metadata, mapped to statifier.driver by the mapping the bridge already has. A durable execution becomes visible the moment the emit sites land, with no second handler table and no second attribute vocabulary. That is st-ADR-0067 decision 1's whole argument, and this package is the reason it was made.

  • session_id maps to statifier.session_id, the same attribute upstream uses. That is what joins family two's events to family one's spans for a consumer that is not relying on ambient context - and it is why family two carries the honest nil rather than a fabricated value on the events emitted before a position is decoded.

  • execution_id maps to statifier_persistence.execution_id, a new attribute in this package's own namespace. Every other measurement and metadata key maps by name into statifier_persistence., as upstream's attribute rule already specifies for its own namespace.

  • Nesting is bridge-owned, three deep. A step span opens and the bridge records it in its own span table, tagged with the emitting pid; the adapter events and the upstream macrostep span land inside it because the bridge parents them from that row, never from the process's ambient OTel context, which it does not read (ots-ADR-0004 decision 4). A job or request span a host or statifier_oban already had open in the same process therefore does not parent the step span - with nothing of the bridge's own open around it, the step span is the root of its own trace. st-ADR-0067 decision 6 describes the shape inside it. No propagation machinery is involved, and no package has to know about another.

  • Resume and cross-step stitching use links, never parenthood, and the position blob carries nothing trace-shaped. ADR-0009 decision 6 settles this. Within a node, consecutive macrosteps of one execution stitch through the bridge's last-span-context table keyed on session_id, exactly as statifier-ex's docs/opentelemetry.md already stitches session macrosteps - the durable case is not special. Across a node or a deploy, where that table misses, a step links to its driving trace if and only if the host attached a st-ADR-0063 caller_context to the driving event; with none, each macrostep is an unlinked trace correlated by statifier.session_id. That is the standard detached case, not an error. Whether this package should reserve a linkage key carrying a W3C traceparent so cross-node resumes link without host cooperation is deferred, with the trigger stated in ADR-0009 decision 6.

  • A child execution links to its parent, and is not parented by it. From [:statifier_persistence, :child, :started] the bridge has parent_execution_id, child_execution_id, invoke_id, child_index, the pinned content_hash and the child's session_id - everything needed for a link, with nothing read out of Execution.Linkage. Parenthood would hold the parent's trace open for the child's whole life.

  • A fan-out is reassembled from :recorded and :settled, not from the parent's span. The invocation is not an interval anything here owns: its children run on N nodes over N intervals, and the only thing that happens at one place and time is each settlement decision. So a bridge reads the invocation as the (parent_execution_id, invoke_id) pair those two events share with :started and :answered, counts the recorded answers against child_count, and reads the last :settled - decision: :answer - as the point the parent moved.

  • trigger is a pass-through string. No event here originates one, but the macrostep events this package emits through Statifier.Telemetry carry upstream's, and it is copied and never validated against an enum. statifier-ex owns that vocabulary and is still growing it (:resume is the most recent addition); a consumer that hardcodes today's value set is wrong on the next release.

  • Trace-off degrades to nothing, structurally. With no bridge attached, :telemetry.execute/3 on an event with no handlers is a lookup and a return. There is no build-time flag, no compile-time removal, and no option to disable emission. Upstream's trace: true gate applies to upstream's nine :trace events and rides the position (st-ADR-0060); nothing in family two scales with microstep count, so it has no counterpart here.

For a host that is not using the bridge

The events are plain :telemetry. A host attaching Telemetry.Metrics gets, with no OpenTelemetry anywhere:

  • a distribution over [:statifier_persistence, :execution, :step, :stop]'s duration, dimensioned by entry and outcome - the durable stepper's own latency, which nothing else measures;
  • a distribution over [:statifier_persistence, :execution, :lock]'s duration and a counter on its outcome - whether the host's concurrency is fighting itself for the same execution;
  • a distribution over [:statifier_persistence, :adapter, :call]'s duration by callback - which storage call is slow, in-memory adapter included;
  • a counter on [:statifier_persistence, :identity, :refused] - the deploy-drift alarm, which should normally be flat at zero;
  • counters on [:statifier_persistence, :execution, :terminated] by driven_by and status, and on [:statifier_persistence, :execution, :discarded] by reason;
  • a counter on [:statifier_persistence, :execution, :migrated] by from_content_hash and to_content_hash - how many executions a sweep over the drained query has moved off a chart;
  • a distribution over [:statifier_persistence, :execution, :migrate_batch, :stop]'s duration by dry_run and outcome, and sums of its counts - how long a batch took and how many executions it moved, refused, parked and skipped;
  • a counter on [:statifier_persistence, :execution, :unparked] by content_hash - how many parked executions a host put back to work unmigrated;
  • counters and a count distribution on the child seam - fan-out, refusals, and how much a cascading cancel actually swept.

That is why the contract is defined in events rather than spans, and it is the same reason st-ADR-0062 gave for the bridge being a separate package.