StatifierPersistence

Durable stepper and storage adapters for Statifier.

Documentation lives on hexdocs, including the Surviving a restart guide.

Statifier's pure interpreter contract (machine_state, event -> machine_state, effects) makes a persistence-first execution model possible: load a persisted position, step it, execute the effects, persist. Hosts running charts that span days or survive deploys should not need long-lived Session processes at all - but every host currently hand-rolls the loop, the storage guard, and the crash semantics. This package is that loop, packaged.

Installation

def deps do
  [
    {:statifier_persistence, "~> 0.7"},
    # Optional, for the Postgres adapter:
    {:ecto_sql, "~> 3.10"}
  ]
end

A worked run

A card-processing transaction: authorize it, capture it before its capture window closes, settle it. The whole run is four calls, and no process holds the chart between them.

alias Statifier.{Chart, Event, Machine, MachineState}
alias Statifier.Invoke.Types, as: InvokeTypes
alias StatifierPersistence.{Runs, Storage}

source = """
<scxml xmlns="http://www.w3.org/2005/07/scxml" version="1.0" initial="authorizing">
  <state id="authorizing">
    <invoke type="myapp:authorize" id="authorize"/>
    <transition event="done.invoke.authorize" target="awaiting_capture"/>
  </state>
  <state id="awaiting_capture">
    <transition event="capture.requested" target="settling"/>
  </state>
  <state id="settling">
    <transition event="ack" target="settled"/>
  </state>
  <final id="settled"/>
</scxml>
"""

Compile the chart once and store its bytes under its own content hash. Nothing is keyed by a name you choose: the identity comes off the compiled Machine, which is what makes the guard unskippable.

{:ok, machine} = Statifier.compile(source)
{:ok, chart_blob} = Chart.to_binary(machine)

{:ok, store} = Storage.new(StatifierPersistence.Storage.InMemory, [])
:ok = Storage.save_chart(store, machine, chart_blob)

Every effect a step emits reaches your host through one seam - a module implementing StatifierPersistence.Executor, or an arity-2 fun. Effects arrive one at a time, in list order, as {tag, payload} tuples. This one does the least a real host could do with an outbound authorization:

executor = fn
  {:invoke, %Statifier.Effect.Invoke{type: "myapp:authorize"} = invoke}, ctx ->
    # your own gateway call, keyed for idempotency by run and invocation
    MyApp.Payments.authorize(ctx.run_id, invoke.invoke_id)
    :ok

  _effect, _ctx ->
    :ok
end

opts = [executor: executor, invoke_types: InvokeTypes.new(types: ["myapp:authorize"])]

create/4 initializes the chart, hands the resulting effects to the executor, and persists the quiescent position under a run id you choose

• here the transaction's own key:

{:ok, run, state} = Runs.create(store, "txn_01H8", machine, opts)
#=> run.status == :active, active leaf state "authorizing"

Each later event is one step/5: liveness check, guarded load, step, effects out through the seam, persist. Between calls there is no live process and no in-memory position - only the run record.

{:ok, run, state} =
  Runs.step(
    store,
    "txn_01H8",
    machine,
    Event.external("done.invoke.authorize", invokeid: "authorize"),
    opts
  )

#=> run.status == :active, active leaf state "awaiting_capture"

Across a restart

Nothing above kept state in the beam, so a deploy in the middle of the run changes nothing about how it continues. Given only the run id, fetch the record, fetch the chart bytes it names, and recompile:

{:ok, record} = Storage.fetch_run(store, "txn_01H8")
{:ok, %{chart_blob: blob}} = Storage.fetch_chart(store, record.content_hash)
{:ok, rebooted} = Chart.from_binary(blob)

rebooted is compiled afresh from the stored bytes, not carried over from before the restart, and it is what makes the stored position readable again: Statifier interns state ids to indices at compile time, so a position is only meaningful against the exact chart revision that produced it. The identity guard enforces that on every load. Step a run with a machine compiled from a changed chart and it refuses with {:error, {:identity_mismatch, stored, supplied}} rather than silently resuming the wrong configuration.

{:ok, run, state} =
  Runs.step(store, "txn_01H8", rebooted, Event.external("capture.requested"), opts)

#=> run.status == :active, active leaf state "settling"

{:ok, run, state} = Runs.step(store, "txn_01H8", rebooted, Event.external("ack"), opts)
#=> run.status == :completed, no active leaf states

:completed is reached only by the chart reaching a final state - the lifecycle consumes the interpreter's :done itself and never hands it to your executor. Runs.fail/4 is the one host-driven terminal transition, and a step delivered to a terminal run comes back {:discarded, run} rather than raising.

To read the configuration back as state ids, as the snippets' comments show it:

state
|> MachineState.active_leaf_states()
|> Enum.map(&Machine.id(state.machine, &1))
|> Enum.sort()

What each module is for

Two things the loop deliberately does not do. Effect delivery is at-least-once: a crash between step and persist re-drives the same event and re-emits the same effects with identical deterministic keys, and the loop never dedupes - idempotency on that key is yours. And a resumed run restores position, not liveness: pending timers and in-flight invocations are re-established by the host, from its own durable rows. Surviving a restart walks a demo embedder through both.

Driving a chart that calls out

Runs steps a run once. A chart that invokes a service is not finished when that step returns - it is waiting for an answer it cannot fetch for itself, and every host that has embedded this package has written the same loop on top. StatifierPersistence.Driver is that loop:

driver =
  StatifierPersistence.Driver.new(store, machine,
    dispatch: fn type, params, _context -> MyApp.perform(type, params) end,
    effects: fn effect, _context -> MyApp.Timers.consume(effect) end,
    invoke_types: Statifier.Invoke.Types.new(types: ["myapp:authorize"]),
    serialization: {MyApp.RunLock, MyApp.RunLock}
  )

{:ok, run, state} = StatifierPersistence.Driver.create(driver, run_id)
{:ok, run, state} = StatifierPersistence.Driver.send_event(driver, run_id, Statifier.Event.external("go"))

One call is one durable step, every effect through your effects: executor, every <invoke> through your dispatch: fun inside that same step, and then one further durable step per answer until the chart rests. {:ok, donedata} answers done.invoke.<id>; {:error, failure} answers error.communication.invoke.<id> with Statifier.Session.failed_invocation/3's own reason/attempts/detail payload, and means permanently failed rather than "retry".

Both events are built field for field from the two doors Statifier.Session gives a handler-backed invocation's host, origin and origintype included, so the same chart sees the same event whether it runs in a session or out of storage. That is asserted rather than claimed: test/statifier_persistence/driver_session_conformance_test.exs answers one document both ways and compares the _event each chart saw.

An answer whose invocation the chart has since cancelled is dropped, per spec 6.4.3, and a chart whose answer re-arms its own call is bounded by max_turns: rather than driven forever.

Fanning one invocation out over N children

A durable subchart is one child per <invoke>, created inside the parent's own step. An <invoke> that maps over a list is N of them, and N creates cannot hold the parent's exclusion, so the children are started afterwards - one call per child, from whatever job picks it up:

StatifierPersistence.Driver.start_child_at(driver, parent_run_id, effect, index, count,
  policy: :all
)

effect is the resolved Statifier.Effect.Invoke (or the whole {:start_child, resolved, {:invoke, invoke}} instruction), index is the child's 0-based position, and count is N. The call is idempotent on the child's derived run id, so a re-delivered start adopts the child it already created instead of making a second one. Scheduling those calls is a job runner's business, not this package's.

Each child then runs as an ordinary run. When one reaches a terminal status its answer is stored on its own run record, and a settlement section under the parent's exclusion asks - through an indexed status projection, never a listing of whole records - whether all N have. Only the settlement that finds them all terminal assembles the dense, index-ordered list and answers the parent's ordinary door, once:

[
  %{"index" => 0, "status" => "completed", "donedata" => %{"id" => "acct_1"}},
  %{"index" => 1, "status" => "failed", "failure" => %{"reason" => "declined", ...}},
  %{"index" => 2, "status" => "cancelled"}
]

policy: :first_error cancels the rest as soon as one child fails: the started siblings through the cascading cancel, and the ones whose start job has not run yet through the child_canceller: seam, which is handed the parent run id, the invocation id, and the indices with no run. Both kinds read "cancelled" at their index in the same list.

An adapter that cannot store a child's answer, or cannot answer the status projection, is refused at open - a child whose invocation could never be settled is not started. On the Ecto adapter both arrive with the V03 migration.

Status

Early, under active development, and the API is not frozen before 1.0. The storage-adapter behaviour with its identity guard, the in-memory reference adapter, the run lifecycle and executor seam, per-run serialization, and the Ecto layer (configurable keys/tables, versioned migrations, and the Postgres adapter below) all exist and are conformance-tested.

The Ecto adapter

Configure a persistence module on your own repo once, and migrate:

defmodule MyApp.Persistence do
  use StatifierPersistence.Ecto, repo: MyApp.Repo
end

defmodule MyApp.Repo.Migrations.AddStatifierPersistence do
  use Ecto.Migration
  def up, do: StatifierPersistence.Ecto.Migrations.up(for: MyApp.Persistence)
  def down, do: StatifierPersistence.Ecto.Migrations.down(for: MyApp.Persistence)
end

One migration covers every version of the package DDL on a fresh database. If you already ran that migration when this package shipped only V01, pick the later versions up with a second ordinary migration rather than re-running the first:

defmodule MyApp.Repo.Migrations.AddStatifierPersistenceRunMetadata do
  use Ecto.Migration
  def up, do: StatifierPersistence.Ecto.Migrations.up(for: MyApp.Persistence, from: 2)
  def down, do: StatifierPersistence.Ecto.Migrations.down(for: MyApp.Persistence, version: 2)
end

then build the guarded store the rest of the package works through:

{:ok, store} =
  StatifierPersistence.Storage.new(
    StatifierPersistence.Storage.Ecto,
    persistence: MyApp.Persistence
  )

The adapter passes the same conformance suite the in-memory reference does (StatifierPersistence.Testing.StorageConformance - point it at your own adapter to hold it to the identical bar), stores engine identities verbatim, and implements the optional per-run lock_run/3 as a transaction-scoped advisory-plus-row lock (ADR-0004 as amended). In your test suite, pass sandbox: true so each test runs in its own Ecto.Adapters.SQL.Sandbox checkout via the adapter's isolate/1.

Upgrading to V03 before deploying 0.7.0

0.7.0 needs V03 of the package DDL, and the order matters: run the migration first, then deploy the new code. outcome_blob is an unconditional field on the generated runs schema, so 0.7.0 against a V02 database fails on every query that touches the runs table, not only on the fan-out write that introduced the column. The reverse order is safe: V03 on a database still served by 0.6.x adds a column nobody writes and an index nobody's query needs yet.

A host already on V02 picks V03 up with an ordinary migration of its own:

defmodule MyApp.Repo.Migrations.AddStatifierPersistenceOutcomeBlob do
  use Ecto.Migration
  def up, do: StatifierPersistence.Ecto.Migrations.up(for: MyApp.Persistence, from: 3)
  def down, do: StatifierPersistence.Ecto.Migrations.down(for: MyApp.Persistence, version: 3)
end

V03 does two things, and only one of them is cheap.

The outcome_blob column is a nullable :binary added to the runs table. Postgres adds a nullable column with no default as a catalog-only change, so this part is fast whatever the table's size. It takes the configured :blob_type with the other three blob columns, so a :blob_type whose underlying database type is not binary needs the same hand-written ALTER this README's encryption section already describes for those three - now for four columns, not three.

The metadata GIN index is the part to plan for. up/1 issues a plain CREATE INDEX, not CREATE INDEX CONCURRENTLY: it takes a SHARE lock on the runs table for the whole build, which blocks every INSERT, UPDATE and DELETE against that table until the index is finished. Reads are unaffected. On a small or idle runs table this is imperceptible. On a large one it is an outage of every write the runs table takes - which, for a host stepping runs durably, means every step of every run.

How long that is depends on the row count, the width of the metadata maps, and the server, so measure rather than guess (sp-461 is this package's own measurement issue if you want a number to compare against). A host with millions of runs rows should treat this as a scheduled maintenance step rather than a deploy-time migration - or write V03's two statements out by hand instead of calling the helper, so the index can be built concurrently:

defmodule MyApp.Repo.Migrations.AddStatifierPersistenceOutcomeBlob do
  use Ecto.Migration

  @disable_ddl_transaction true
  @disable_migration_lock true

  def up do
    alter table("runs") do
      add(:outcome_blob, :binary, null: true)
    end

    create(
      index("runs", ["metadata jsonb_path_ops"],
        using: "GIN",
        name: :runs_metadata_gin_index,
        concurrently: true
      )
    )
  end
end

substituting your configured table name and prefix throughout; the index name must stay <runs table>_metadata_gin_index, which is what V03's down/1 drops. A concurrent build does not block writes and takes longer, and it cannot run inside a transaction, which is what the two module attributes are for. A host that goes this route has reached V03 without calling up(from: 3) and should not call it afterwards: V03's create/1 is a plain create, not create_if_not_exists, so a second run fails on the index that is already there.

The index is not optional in effect: without it, every fan-out child completion asks whether its N siblings are terminal with a jsonb containment query, and each one is a sequential scan of the whole runs table.

On an Ecto adapter that is not Postgres, the index is skipped. GIN and jsonb_path_ops are Postgres spellings, so up/1 creates the index only when the migration's repo runs on Ecto.Adapters.Postgres, and down/1 drops it under the same condition. Everything else in V03 - the outcome_blob column included - is created on every adapter, which is what lets a SQLite host run this package's DDL at all. (In 0.7.0 it could not: the index raised, the whole migration rolled back, and the column went with it. Fixed in 0.7.1.)

What is skipped with the index is what the index served. Both metadata queries this package issues - StatifierPersistence.Storage.Ecto.list_runs_by_metadata/2 and the status projection list_run_states_by_metadata/2 - are jsonb containment SQL, which a non-Postgres backend does not parse. So on such an adapter the Ecto adapter declares no metadata support: a metadata: map at create is refused with {:error, :metadata_unsupported}, the two listings refuse with {:error, :child_listing_unsupported} and {:error, :run_states_unsupported}, and a durable subchart or a fan-out over that store is refused at open rather than started and left with children nothing can settle. Storing, loading, stepping and resuming runs are unaffected. Per-run locking is a separate Postgres-only surface - lock_run/3 is pg_advisory_xact_lock plus SELECT ... FOR UPDATE - and is tracked in sp-5lm.

Listing runs by host scope

A run record carries engine identities and opaque blobs. Nothing on it answers the question a multi-tenant host asks first - "list the runs for scope X" - so ADR-0006 adds one optional, opaque metadata map to a run, stored beside it and handed back unchanged.

Take a card-processing host running a myapp:authorize / myapp:capture chart, one run per payment attempt, and a support screen that lists every run for one processor account. Tag the run at create with the account ids the host already keys its own tables by:

{:ok, run, _machine_state} =
  StatifierPersistence.Runs.create(store, payment_id, machine,
    executor: MyApp.Executor,
    metadata: %{
      "tenant_id" => "acct_01H8X",
      "processor_account_id" => "pacct_4471"
    }
  )

and read them back with an equality match on every pair:

{:ok, runs} =
  StatifierPersistence.Storage.Ecto.list_runs_by_metadata(store.opts, %{
    "processor_account_id" => "pacct_4471"
  })

Equality on all given pairs is the whole query surface: no ranges, no partial matches, no ordering guarantee. Anything richer is a query you write against your own column - the table name is yours to configure, so that is a supported thing to do. The V02 migration adds the column as nullable jsonb with no index of its own, because which pairs you query by is your call. V03 (above) adds the one containment index this package's own settlement query needs, a GIN jsonb_path_ops index on the whole column; an expression index on particular keys, or the wider jsonb_ops operator set, is still yours to add when the volume asks.

Two rules come with it.

Identities only, never personal data. Keys and values are host identities - a tenant id, a subject-entity id, a correlation id - and never a name, an email address, a postal address, a card number, or any other personal or cardholder data. This is a rule of the contract, not advice: :blob_type encryption (below) covers the three blob columns and does not reach this one, so anything you file here is at rest in the clear no matter how the blobs are configured. The map is opaque to this package by design, so nothing here can inspect a value and reject it - the rule is kept by you.

An adapter may refuse it. An adapter that cannot store the map refuses a non-empty one at the create with {:error, :metadata_unsupported}, so you learn on the first call rather than finding a silently dropped scope later. An empty or absent map is never refused. The shipped in-memory and Ecto adapters both support it; a third-party adapter that does not is still conformant, and the conformance suite tests both answers. The Ecto adapter refuses at the same point for a value jsonb cannot hold - a tuple, an atom, a pid, or a binary that is not valid UTF-8 - rather than storing something that is not what you handed it. The map is write-once: it is set at create and a later step or abandonment carries it forward untouched.

Encrypting the blob columns

use StatifierPersistence.Ecto hard-codes :binary for its three blob columns (identity_blob, chart_blob, position_blob) by default - plain bytea, byte-identical round trip, nothing extra. Pass :blob_type to put a custom Ecto type on those three columns instead, and encryption at rest needs no wrapping adapter:

defmodule MyApp.Persistence do
  use StatifierPersistence.Ecto,
    repo: MyApp.Repo,
    blob_type: MyApp.EncryptedBlob
end

:blob_type accepts a bare module implementing Ecto.Type, or a {module, opts} tuple for an Ecto.ParameterizedType. It reaches only those three columns: keys and lookup columns (content_hash, session_id, run_id, status, failure) always stay plain text, because the identity guard and the unique indexes depend on reading them back verbatim.

The shape a production MyApp.EncryptedBlob needs is a vault-backed or envelope-encrypting Ecto.Type - dump/1 encrypts on the way in, load/1 decrypts on the way out. This package takes no position on which key-management scheme backs it; that choice belongs to the host. To prove the shape without any encryption dependency, here is a minimal Ecto.Type that reversibly transforms every byte (not encryption - a stand-in to show the wiring):

defmodule MyApp.ReversibleBlob do
  use Ecto.Type

  @mask 0xA5

  def type, do: :binary
  def cast(binary) when is_binary(binary), do: {:ok, binary}
  def cast(_other), do: :error
  def dump(binary) when is_binary(binary), do: {:ok, transform(binary)}
  def dump(_other), do: :error
  def load(binary) when is_binary(binary), do: {:ok, transform(binary)}
  def load(_other), do: :error

  defp transform(binary) do
    for <<byte <- binary>>, into: <<>>, do: <<Bitwise.bxor(byte, @mask)>>
  end
end

The shipped V01 migration always emits :binary (bytea) for the three blob columns and does not read :blob_type. A :blob_type whose underlying database type is still binary - an envelope-encrypting type that dumps to and loads from raw bytes, like the sketch above - needs no DDL change. A :blob_type that dumps to a different underlying type (text, jsonb, a Postgres domain) needs you to alter those three columns yourself; the migrations helper does not do it for you.

Running the tests

The suite includes database-backed tests against a real Postgres server - ADR-0005 rejects a skip tag for when one is absent, so mix quality and mix test both need one reachable. Start it once with:

docker compose up -d db

which brings up postgres:17 on localhost:5432 with user/password postgres. Override host, port, user, password, or database name with the PGHOST, PGPORT, PGUSER, PGPASSWORD, and PGDATABASE env vars (see config/test.exs for the defaults) if a server is already running elsewhere.

Surviving a restart

docs/restart-demo.md walks through the demo embedder that drives this package's whole surface across a simulated restart with no Session process: persist mid-run with a pending durable timer and an in-flight async invocation, drop everything volatile, cold-boot from the run id alone, and finish with zero duplicate side effects and a replay that reproduces the path. The executable version lives in test/statifier_persistence/demo/restart_demo_test.exs (and its Postgres variant beside it).

The contract this package builds on

The persisted-position story is already specified upstream, and this package is one consumer of it rather than the definition of it:

• docs/persistence.md in statifier-ex covers what MachineState contains, the interned-index hazard, chart identity, and the resume recipe.
• ADR-0052 there records the rules: a persisted position is only meaningful against the exact chart revision that produced it, so every load is guarded by the Machine identity / content-hash. Loading a position against the wrong revision does not error - it silently resumes the wrong configuration.
• ADR-0060 records the resume API: the :resume option on Session.start_link/2, the pure-core rehydration path, and what a resume deliberately does not restore (in-flight delayed-send timers and live invoked children).

Read all three before adding code here.

Scope

In scope:

• A storage-adapter behaviour: save/load of MachineState snapshots (or Recordings), guarded by the Machine identity so a position can never be loaded against the wrong chart revision.
• Run lifecycle as a library: create/step/complete/fail, with a serialization guarantee per run so concurrent event deliveries to one run are ordered.
• The load -> handle_event -> execute effects -> persist loop, with effect execution delegated to the host.
• An Ecto adapter shipping schemas and migrations for chart definitions, versions, and runs; the host supplies the Repo and any tenancy columns.

Out of scope: domain actions, authoring UI, and job scheduling - statifier_oban owns timers and async work.