StatifierPersistence.Driver (StatifierPersistence v0.8.0)

Copy Markdown View Source

Run-to-quiescence over StatifierPersistence.Runs: the loop that answers the chart's <invoke> calls and keeps stepping until it stops asking.

StatifierPersistence.Runs steps a run once. That is the durable unit and it is deliberately small - load, step, hand the effects to an executor, persist - but it is not what a host wants to call. A chart that invokes a service is not finished when the step that emitted the <invoke> returns: it is waiting for an answer it has no way to fetch for itself. Every host that has embedded this package has written the same loop on top - step, collect the calls, perform them, feed each answer back, step again - and every hand-written copy of it is a place where a durable run can quietly stop meaning what the same chart means under Statifier.Session.

This module is that loop, with the event construction taken from Statifier.Session's own rather than reinvented beside it.

What one drive does

One call to create/3 or send_event/4 is:

  1. one StatifierPersistence.Runs entry point - the durable step, with the run's whole fetch-to-persist tail inside its serialization strategy;
  2. every non-lifecycle effect through the host's :effects executor, in list order, exactly as Runs already hands them over;
  3. every {:invoke, _} effect also through the host's :dispatch fun, synchronously, inside that same tail;
  4. after the tail has returned - never inside it - one further Runs.step/5 per answer, in the order the calls were made, each of which can produce answers of its own;
  5. repeat from 4 until no answer is left. The result is the last step's own result.

The ordering in 3 and 4 is forced rather than stylistic. Dispatch runs inside the tail because a call the chart made and a call the host performed have to be the same event in the same durable step; stepping runs outside it because the tail is already inside the run's serialization strategy, and a step issued from within would ask for exclusion its own caller is holding.

Answers are events, and they are Session's events

Statifier.Session gives a handler-backed invocation's host exactly two doors: Statifier.Session.done_invocation/3 and, per st-ADR-0068, Statifier.Session.failed_invocation/3. Both build an external event and enqueue it. This module builds the same two events, field for field, from the same Statifier.Evaluator.SystemVariables writers:

  • {:ok, donedata} from :dispatch becomes done.invoke.<invoke_id>, carrying donedata as its data, its invokeid, and C.1's origin/origintype pair.
  • {:error, failure} becomes error.communication.invoke.<invoke_id>, whose data is st-ADR-0068's three string keys - "reason" (default "unknown"), "attempts" and "detail" (both :undefined when the host supplies none, never nil) - built from the same failure keyword list failed_invocation/3 reads.

origin is #_scxml_<session id>, and the session id comes from the run's own persisted _sessionid (spec 5.10, st-ADR-0008), which Statifier.Position carries in the datamodel across a restart. A resumed run therefore answers with the same origin the run started with, on a node that has never seen it before.

The error arm is permanent failure, in st-ADR-0068's sense: the host's retry policy is exhausted and no done.invoke will follow. A transient failure is the host's to retry inside :dispatch before answering, not something to report to the chart.

What is not answered

A buffered answer is dropped rather than delivered when its invocation is no longer live by the time its turn comes - spec 6.4.3's drain-time discard, read off machine_state.active_invocations the same way Statifier.Interpreter reads it. A step that cancelled an invocation therefore takes that invocation's answer with it, which is what a session does.

<invoke type="scxml"> is handed to :dispatch like any other type.

Durable subcharts

A durable driver holds no child session between steps, so a subchart cannot be answered the way Statifier.Session answers one - by starting and holding a child session in memory. :dispatch answers a subchart instead: {:start_child, invoke, {:invoke, invoke}}, the same instruction Statifier.Session.Effects plans and that the built-in Statifier.Invoke.Handler.Scxml and StatifierBlocks.Runtime.Subchart both emit, unchanged, whichever session executes it. This module is the durable executor for it (ADR-0008 decision 3): it resolves and creates the child as an ordinary run, linked to this invocation under a reserved-namespace pin of the child's own chart identity (StatifierPersistence.Run.Linkage, ADR-0008 decision 2), drives that run to its own quiescence through this same loop - so a child that itself invokes a grandchild is handled with no extra code - and then answers :pending under ADR-0007 decision 1, exactly as any other asynchronous call does: the parent rests with the invocation live and no process holding it. The instruction is never renamed or reshaped, which is what makes a chart portable between the in-memory and durable paths.

Invocations answered later

A host whose service does not answer inside the drive - an enqueued job, a webhook, anything that outlives the process that started it - answers :pending from :dispatch instead. The call has been started; nothing is buffered for it; the drive rests and the position persists with the invocation still live in machine_state.active_invocations. There is no process holding the run in the meantime, which is the point: the run can wait days and survive a deploy.

The answer arrives later through done_invocation/5 or failed_invocation/5 - the two doors Statifier.Session gives a live session's host, on the durable path and keyed by the same invoke_id. They build the same two events the in-drive path builds and drive the run from them, so a chart cannot tell which way its answer came.

The cancel-versus-completion race

An invocation the chart cancels while its call is still running has an answer coming for something that is no longer live - across a restart, on a node that has never seen the run. The liveness read that settles it is active_invocations, which Statifier.Position persists and Statifier.Interpreter.ExitEntry empties when the invoking state is exited, and it is taken inside the run's serialization strategy: the door hands StatifierPersistence.Runs.step/5 an event builder rather than an event, and the builder reads the loaded position under the same exclusion the step itself holds. A check taken before the call would leave a window for a cancel to land between the read and the step.

A cancelled invocation's answer is {:discarded, run} - spec 6.4.3's discard again, the same rule the in-drive loop applies at drain time - and the chart never sees it.

This makes re-entry idempotent for the ordinary chart, which transitions out of the invoking state on its answer: the second delivery finds the invocation gone. It does not make it idempotent for a chart that stays in the invoking state after answering, because the core removes an entry from active_invocations on exit and on nothing else. That is the in-drive path's behavior too, not something the doors introduce, and it is where a host's own delivery-once discipline belongs (ADR-0007).

Bounding the loop

A chart whose answer re-arms the call it answered would drive forever. :max_turns (default 1000) bounds the answer-fed steps in one drive and returns {:error, {:turns_exhausted, max_turns}} when it is reached. The run is durable and quiescent at that point - every step that ran, persisted - so the error names a loop this driver refused to keep turning, not a lost position.

Example

driver =
  StatifierPersistence.Driver.new(store, machine,
    dispatch: fn type, params, _context -> MyApp.perform(type, params) end,
    effects: fn effect, _context -> MyApp.Timers.consume(effect) end,
    invoke_types: Statifier.Invoke.Types.new(types: ["myapp:authorize"]),
    serialization: {MyApp.RunLock, MyApp.RunLock}
  )

{:ok, run, machine_state} = StatifierPersistence.Driver.create(driver, run_id)

{:ok, run, machine_state} =
  StatifierPersistence.Driver.send_event(driver, run_id, Statifier.Event.external("go"))

Summary

Types

How this driver reaches a chart it does not hold: answering a durable subchart's parent, whose chart is not this driver's own machine (ADR-0008 decision 3). content_hash is the parent run's own, read off its stored record.

How this driver reaches the scheduler that holds a fan-out's not-yet- started children (sp-t57, ruling C9; sob-q3y implements it).

Performs one <invoke> and answers it - synchronously inside the durable step that emitted it, or later through this module's re-entry doors.

What dispatch/0 receives as its third argument: the executor's own context - the run id and the chart's content hash - plus invoke_id, this invocation's id, and invoke, the effect payload being dispatched.

What one drive returns: the last durable step's own result.

t()

Functions

Answers child_run_id's parent with its completion or permanent failure (ADR-0008 decision 3) - a separate drive under the parent's own exclusion, driver.machine must be the parent's chart.

Creates the run under run_id and drives it to quiescence.

Answers a :pending invocation with donedata and drives the run to quiescence.

done_invocation/5's failing counterpart: answers a :pending invocation with a permanent failure and drives the run to quiescence.

Builds a driver over store and machine.

The "find my parent" query: reads run_id's own stored linkage, one key read off its fetched record (StatifierPersistence.Run.Linkage, ADR-0008 decision 2).

Delivers one external event to the run under run_id and drives it to quiescence.

Starts child index of count for parent_run_id's <invoke> - the public start-with-index door a scheduler drives a fan-out through (sp-t57, ruling C4; mirrors sob-q3y).

Types

chart_resolver()

@type chart_resolver() :: (content_hash :: String.t() ->
                       {:ok, Statifier.Machine.t()} | :error)

How this driver reaches a chart it does not hold: answering a durable subchart's parent, whose chart is not this driver's own machine (ADR-0008 decision 3). content_hash is the parent run's own, read off its stored record.

child_canceller()

@type child_canceller() :: (parent_run_id :: StatifierPersistence.Runs.run_id(),
                      invoke_id :: String.t(),
                      unstarted_indices :: [non_neg_integer()] ->
                        :ok | {:error, term()})

How this driver reaches the scheduler that holds a fan-out's not-yet- started children (sp-t57, ruling C9; sob-q3y implements it).

first_error cancels the invocation's remaining children. The ones that already have a run are this package's own to cancel, through StatifierPersistence.Runs.cascade_cancel/3. The ones whose start job has not run yet have no run record at all, so nothing here can see them, let alone cancel them - only the scheduler that enqueued their jobs can. This is the call that asks it to.

It receives the parent's run id, the invocation id, and the indices in 0..child_count - 1 that produced no run record - the exact set of start jobs to cancel, computed inside the settlement section under the parent's exclusion. {:error, reason} fails the settlement rather than answering a dense list whose cancelled entries it could not vouch for.

Defaults to nil, "this driver cancels no start jobs": a host with no scheduler starts no fan-out, and a :first_error settlement over a fully-started fan-out needs none either, since every index already has a run for the cascade to reach.

dispatch()

@type dispatch() :: (type :: String.t() | nil,
               params :: term(),
               context :: dispatch_context() ->
                 {:ok, term()}
                 | {:error, keyword()}
                 | :pending
                 | {:start_child, Statifier.Effect.Invoke.t(),
                    {:invoke, Statifier.Effect.Invoke.t()}})

Performs one <invoke> and answers it - synchronously inside the durable step that emitted it, or later through this module's re-entry doors.

Receives the element's own type and resolved params (Statifier.Effect.Invoke.t/0's fields) plus a dispatch_context/0, whose :invoke key carries that whole payload for a host that needs a field the two arguments do not name - src being the one a chart resolver keys on. {:ok, donedata} answers done.invoke.<invoke_id> with donedata; {:error, failure} answers error.communication.invoke.<invoke_id> with st-ADR-0068's failure keyword list (:reason, :attempts, :detail), and means permanently failed, not "try again".

:pending is the asynchronous arm: the call has been started and will be answered later, by done_invocation/5 or failed_invocation/5, from whatever process - or whatever node, after whatever restart - eventually has the result. Nothing is buffered for it and the drive rests, so the run reaches quiescence and persists with the invocation still live.

{:start_child, invoke, {:invoke, invoke}} means: start this chart as the child of this invocation. It is Statifier.Session.Effects' own instruction, emitted unchanged by StatifierBlocks.Runtime.Subchart and by the built-in Statifier.Invoke.Handler.Scxml - this module executes it where Statifier.Session executes it in-memory, which is what makes a chart portable between the two (ADR-0008 decision 3). It is never renamed or reshaped, and a host never has to build this tuple itself: a subchart handler returns it unchanged from what it received, which is what dispatch_context/0's :invoke key makes literally true.

dispatch_context()

@type dispatch_context() :: %{
  run_id: String.t(),
  content_hash: String.t(),
  invoke_id: String.t(),
  invoke: Statifier.Effect.Invoke.t()
}

What dispatch/0 receives as its third argument: the executor's own context - the run id and the chart's content hash - plus invoke_id, this invocation's id, and invoke, the effect payload being dispatched.

invoke_id is here and not in StatifierPersistence.Executor.context/0 because it is not a property of the run or the step: it names one <invoke>, and only the dispatch fun is called per invocation. It is what an asynchronous host keys its job by, and the same string done_invocation/5 and failed_invocation/5 take back.

invoke is the whole Statifier.Effect.Invoke.t/0 this dispatch is for, and it is here for the same reason: it is a property of the one <invoke>, not of the run or the step. type and params are handed over as their own arguments because they are what an ordinary host acts on; the rest of the element - src above all, and content, autoforward, and the counters with it - reaches a host that needs it only through this key. src is spec 6.4's URI attribute, which the core never dereferences (st-ADR-0031): a host that resolves a chart by document id reads context.invoke.src, and a subchart handler that answers {:start_child, invoke, {:invoke, invoke}} returns the payload it was handed rather than synthesising one from what it happened to know (ADR-0007 decision 5's amendment, ADR-0008 decision 3).

result()

@type result() ::
  {:ok, StatifierPersistence.Run.t(), Statifier.MachineState.t()}
  | {:discarded, StatifierPersistence.Run.t()}
  | {:error,
     StatifierPersistence.Runs.error() | {:turns_exhausted, pos_integer()}}

What one drive returns: the last durable step's own result.

{:ok, run, machine_state} for a run that reached quiescence with nothing left to answer, {:discarded, run} for an event delivered to a terminal run, and the error arms of StatifierPersistence.Runs plus this module's own {:turns_exhausted, max_turns}.

t()

@type t() :: %StatifierPersistence.Driver{
  chart_resolver: chart_resolver() | nil,
  child_canceller: child_canceller() | nil,
  dispatch: dispatch(),
  effects: StatifierPersistence.Executor.t() | nil,
  invoke_types: Statifier.MachineState.invoke_types(),
  machine: Statifier.Machine.t(),
  max_turns: pos_integer(),
  serialization: {module(), term()} | nil,
  store: StatifierPersistence.Storage.t()
}

Functions

answer_parent(driver, child_run_id, donedata_or_failure)

@spec answer_parent(
  driver :: t(),
  child_run_id :: StatifierPersistence.Runs.run_id(),
  donedata_or_failure :: {:done, term()} | {:failed, keyword()}
) ::
  result() | :ok | :no_parent | {:error, StatifierPersistence.Storage.error()}

Answers child_run_id's parent with its completion or permanent failure (ADR-0008 decision 3) - a separate drive under the parent's own exclusion, driver.machine must be the parent's chart.

donedata_or_failure is {:done, donedata} or {:failed, failure}. Reads child_run_id's own linkage through parent_link/2 first: :no_parent is a no-op answering :no_parent, so this is safe to call on any run id, linked or not.

A child of a fan-out answers no parent here. Its linkage carries a child_count, so this call settles instead - records the child's own answer and, if it is the last, assembles the invocation's dense list and answers the parent's door once - and returns :ok. The routing is here and not only on the automatic path because a host driving the doors itself must not be able to bypass a settlement by calling this function: answering a fan-out's parent with one child's donedata would complete the whole map block on the first child to finish.

Public so a host with no chart_resolver: can call it explicitly with a driver built over the parent's own chart - the same construction the automatic path (wired into create/3, send_event/4, done_invocation/5 and failed_invocation/5) uses once its chart_resolver: has resolved one. The parent's answer is done_invocation/5 or failed_invocation/5 under the parent's own exclusion: a parent that has already cancelled the invocation answers {:discarded, _} here, which is ADR-0007 decision 3's mechanism doing its job, not an error.

create(driver, run_id, opts \\ [])

@spec create(
  driver :: t(),
  run_id :: StatifierPersistence.Runs.run_id(),
  opts :: keyword()
) :: result()

Creates the run under run_id and drives it to quiescence.

StatifierPersistence.Runs.create/4 with this driver's executor, then the answer loop. opts takes everything create/4 takes except executor:, which this module supplies - initialize:, metadata:, routes:, and per-call overrides of the driver's own invoke_types: and serialization:.

done_invocation(driver, run_id, invoke_id, donedata \\ nil, opts \\ [])

@spec done_invocation(
  driver :: t(),
  run_id :: StatifierPersistence.Runs.run_id(),
  invoke_id :: String.t(),
  donedata :: term(),
  opts :: keyword()
) :: result()

Answers a :pending invocation with donedata and drives the run to quiescence.

Statifier.Session.done_invocation/3's door on the durable path: it builds the same done.invoke.<invoke_id> event, from the run's own persisted _sessionid, and steps it. invoke_id is the <invoke> element's id - the invoke_id :dispatch was handed in its dispatch_context/0.

Answering an invocation the chart has since cancelled is {:discarded, run}, spec 6.4.3's discard, decided from the loaded position inside the run's serialization strategy (the moduledoc's cancel-versus-completion section). So is answering a terminal run.

The answer can re-arm calls of its own; they are dispatched and driven exactly as send_event/4 drives them, :pending included.

opts takes what send_event/4 takes.

failed_invocation(driver, run_id, invoke_id, failure \\ [], opts \\ [])

@spec failed_invocation(
  driver :: t(),
  run_id :: StatifierPersistence.Runs.run_id(),
  invoke_id :: String.t(),
  failure :: keyword(),
  opts :: keyword()
) :: result()

done_invocation/5's failing counterpart: answers a :pending invocation with a permanent failure and drives the run to quiescence.

Statifier.Session.failed_invocation/3's door on the durable path, building the same error.communication.invoke.<invoke_id> event from st-ADR-0068's failure keyword list (:reason, :attempts, :detail). Permanent in that record's sense: the host's retry policy is exhausted and no done.invoke will follow. A transient failure is the host's to retry before answering, not something to report to the chart.

Discards, re-armed calls and opts are done_invocation/5's.

new(store, machine, opts)

@spec new(
  store :: StatifierPersistence.Storage.t(),
  machine :: Statifier.Machine.t(),
  opts :: keyword()
) :: t()

Builds a driver over store and machine.

opts:

  • dispatch: (required) - the dispatch/0 fun every <invoke> is performed through.
  • effects: - a StatifierPersistence.Executor.t/0 handed every non-lifecycle effect before the invoke dispatch, for the effects the host observes or persists itself (a <send delay=...> becoming a durable timer, a trace becoming a feed row). Defaults to nil, "the host wants none of them"; an {:error, reason} from it re-enters the chart as error.communication exactly as it does through StatifierPersistence.Runs directly.
  • invoke_types: - the Statifier.Invoke.Types.t/0 snapshot stamped on every step. A driver-level default rather than a per-call one because the registered set is fixed for a session's lifetime (st-ADR-0051); routes:, which is not, stays per call. Defaults to nil, "the built-in set only".
  • serialization: - the {module, config} per-run strategy every entry point runs inside (ADR-0004 decision 5). Defaults to whatever StatifierPersistence.Runs defaults to, the adapter's own lock_run/3.
  • chart_resolver: - chart_resolver/0, how this driver reaches a chart it does not hold: (content_hash -> {:ok, Statifier.Machine.t()} | :error). It exists for exactly one purpose - answering a durable subchart's parent, whose chart is not this driver's machine (ADR-0008 decision 3). This package cannot supply it: a stored chart_blob is opaque by ADR-0003 decision 1 and nothing here decodes one, so the host that saved the chart is the only party that can compile it. Defaults to nil, "this driver answers no parents" - a host without one calls done_invocation/5 or failed_invocation/5 itself, from parent_link/2 and the drive's own run.donedata or run.failure.
  • child_canceller: - child_canceller/0, how a :first_error settlement reaches the scheduler holding the start jobs of a fan-out's not-yet-started children. Defaults to nil, "this driver cancels no start jobs".
  • max_turns: - the answer-fed steps one drive will take before refusing to take another. Defaults to 1000.

Every one of these except dispatch: may be overridden per call by passing the same key in a create/3 or send_event/4 opts list.

parent_link(store, run_id)

@spec parent_link(
  store :: StatifierPersistence.Storage.t(),
  run_id :: StatifierPersistence.Runs.run_id()
) ::
  {:ok, StatifierPersistence.Run.Linkage.t()}
  | :no_parent
  | {:error, StatifierPersistence.Storage.error()}

The "find my parent" query: reads run_id's own stored linkage, one key read off its fetched record (StatifierPersistence.Run.Linkage, ADR-0008 decision 2).

:no_parent for a run with no linkage - an ordinary run, or a durable subchart child that has none for whatever reason - not a failure: having no parent is an ordinary property of a run.

send_event(driver, run_id, event, opts \\ [])

@spec send_event(
  driver :: t(),
  run_id :: StatifierPersistence.Runs.run_id(),
  event :: Statifier.Event.t(),
  opts :: keyword()
) :: result()

Delivers one external event to the run under run_id and drives it to quiescence.

StatifierPersistence.Runs.step/5 with this driver's executor, then the answer loop. An event delivered to a terminal run is that function's own {:discarded, run}, before any position decode and before any dispatch.

start_child_at(driver, parent_run_id, effect, index, count, opts \\ [])

@spec start_child_at(
  driver :: t(),
  parent_run_id :: StatifierPersistence.Runs.run_id(),
  effect ::
    Statifier.Effect.Invoke.t()
    | {:start_child, Statifier.Effect.Invoke.t(),
       {:invoke, Statifier.Effect.Invoke.t()}},
  index :: non_neg_integer(),
  count :: pos_integer(),
  opts :: [{:policy, StatifierPersistence.Run.Linkage.policy()}]
) :: :ok | {:refused, term()}

Starts child index of count for parent_run_id's <invoke> - the public start-with-index door a scheduler drives a fan-out through (sp-t57, ruling C4; mirrors sob-q3y).

The single-child durable-subchart path creates its child from inside the parent's own step, because there is exactly one and the parent is already exclusive. A fan-out cannot: N children created inside the parent's step would hold the parent's exclusion for N creates. So the parent's step enqueues the fan-out instead, and each child is created later, from whatever job picks it up, through this function. That is the reading of statifier_blocks ADR-0008 decision 4 the fan-out needs: what happens under the parent's exclusion is the enqueue, and the children are created afterwards, idempotently and resumably.

Idempotent, and that is what makes it resumable: the child's run id is StatifierPersistence.Run.Linkage.child_run_id/3 of the same three values, so a re-delivered start job finds the child it already created and adopts it rather than creating a second one - exactly as the single-child path's at-least-once re-drive does.

Arguments

  • driver - any driver over the right store. Its machine is not read: the child's chart comes from effect, and the parent's comes from the chart_resolver: when the settlement answers.
  • parent_run_id - the run whose <invoke> this fans out.
  • effect - the resolved Statifier.Effect.Invoke.t/0, or the whole {:start_child, resolved, {:invoke, invoke}} instruction a subchart handler answers with. The invocation id is read off it, so a caller passes no id separately.
  • index - the child's 0-based position in the list being mapped over.
  • count - N, recorded on every child so a settlement knows how many to wait for.
  • opts - policy: (:all, the default, or :first_error).

index outside 0..count - 1 raises ArgumentError - a caller bug, not a storage event. The check is StatifierPersistence.Run.Linkage.new/6's, which is the one definition site of the linkage's own shape; this function does not repeat it.

Refusals

{:refused, reason}, the same shape and the same telemetry the single-child path's refusal at open uses, with three added arms. An adapter that cannot enumerate children refuses :child_listing_unsupported as it always has; one that cannot store a run's outcome payload refuses :run_outcome_unsupported, and one that cannot answer the indexed status projection refuses :run_states_unsupported. All three are the same principle: a child whose invocation could never be settled is not started. A parent_run_id naming no stored run refuses :run_not_found.