Continuum.Test.Paranoid (continuum v0.5.0)

Copy Markdown View Source

Determinism safety net for the test suite — the --paranoid re-replay mode.

When enabled, every workflow run that reaches a :completed terminal state is re-replayed from its journaled history through Continuum.Test.replay/4. The re-replay asserts an identical (event_type, decoded_payload, command_id) sequence between the original execution and the replay, and that the replay produces the same result. DB-stamped fields (:seq, :inserted_at) are excluded from the comparison.

Enable it for a whole mix test run with the CONTINUUM_PARANOID=1 environment variable (or config :continuum, :paranoid_replay, true). The default is off so ordinary mix test stays fast — the expensive paranoid sweep is meant for the push-to-main CI matrix.

Two surfaces:

  • verify_run!/4 — synchronous, raises on divergence. Call it directly from a test for belt-and-suspenders coverage of a specific run.
  • attach!/0 — installs a telemetry handler that auto-verifies every :completed run. Mismatches are logged and collected; mismatches/0 and reset/0 let an ExUnit.after_suite callback surface them without crashing the engine that emitted the completion event.

Summary

Functions

Assert two journaled histories carry an identical (event_type, decoded_payload, command_id) sequence.

Attach the auto-verify telemetry handler for the [:continuum, :run, :completed] event. Idempotent.

Detach the auto-verify telemetry handler.

Whether paranoid re-replay is enabled for this run.

Collected mismatches found by the auto-verify handler.

Normalize a history for comparison: drop DB-stamped fields.

Reset the collected mismatches.

Re-replay a terminal run's journaled history and assert it is identical.

Functions

assert_histories_match!(original, replayed)

@spec assert_histories_match!([map()], [map()]) :: :ok

Assert two journaled histories carry an identical (event_type, decoded_payload, command_id) sequence.

DB-stamped fields (:seq, :inserted_at) are excluded from the comparison.

attach!()

@spec attach!() :: :ok

Attach the auto-verify telemetry handler for the [:continuum, :run, :completed] event. Idempotent.

detach!()

@spec detach!() :: :ok

Detach the auto-verify telemetry handler.

enabled?()

@spec enabled?() :: boolean()

Whether paranoid re-replay is enabled for this run.

mismatches()

@spec mismatches() :: [map()]

Collected mismatches found by the auto-verify handler.

normalize(history)

@spec normalize([map()]) :: [map()]

Normalize a history for comparison: drop DB-stamped fields.

reset()

@spec reset() :: :ok

Reset the collected mismatches.

verify_run!(workflow_module, input, run_id, opts \\ [])

@spec verify_run!(module(), term(), binary(), keyword()) :: :ok

Re-replay a terminal run's journaled history and assert it is identical.

Loads the run's recorded history, replays the workflow against it, and asserts that:

  • no Continuum.ReplayDriftError is raised (the journaled (event_type, payload, command_id) sequence still matches what the orchestration requests, in order), and
  • the replayed result equals the result the original run recorded.

Raises an ExUnit.AssertionError on divergence. Accepts the same :journal and :instance options as Continuum.Test.replay/4.