Mix.Statifier.Corpus.Runner (Statifier v2.9.0)

Copy Markdown View Source

Runs corpus cases through statifier in the calling process's node, the way a generated test module runs them: each case is one call to Statifier.Testing.Case.test_scxml/4 with the case's source, description, initial configuration and steps, so a case agrees exactly when its generated module would pass.

test_scxml/4 asserts that the active leaf states statifier produces equal the expected ones after initialization and after every step, so a case that agrees has configurations that are what statifier produced; a case that disagrees carries the assertion's own message. Cases run concurrently, as the generated modules do (async: true); a case needing a session (<send>, <invoke>, timers) needs the :statifier application started, which mix statifier.corpus does before it runs any.

A case carrying a host object runs through Mix.Statifier.Corpus.HostCase instead, which registers the case's send types with the session it starts and compares the sends handed to them with the case's expect_sends (ADR-0070 decision 5); when the host object carries declared_events, it also compares Statifier.Chart.check_accepts/2's answer with the case's expect_accepts (ADR-0071 decision 7).

Summary

Types

What running one case found.

Functions

Runs every case, returning {id, outcome} pairs in the order given.

Runs one case through Statifier.Testing.Case.test_scxml/4, or through Mix.Statifier.Corpus.HostCase.run/1 when it carries a host object.

Runs the authored cases the regression ratchet names by paths, each path the case's JSON file (Mix.Statifier.Corpus.Authored.case_path/1), relative to root or already joined to it. Returns {path, outcome} pairs in the order given, reading every authored case under root once. A path that names no authored case disagrees, naming itself: the ratchet never passes a case it cannot find. A malformed case tree is refused as Mix.Statifier.Corpus.Authored.read/1 refuses it.

Starts the :statifier application and places Statifier.Supervisor, the session runtime a case using <send>, <invoke> or a delay runs through (ADR-0027: the library starts no processes of its own). Idempotent: a runtime already placed is left as it is.

Types

outcome()

@type outcome() :: :agree | {:disagree, String.t()}

What running one case found.

Functions

run(cases)

@spec run(cases :: [map()]) :: [{String.t(), outcome()}]

Runs every case, returning {id, outcome} pairs in the order given.

run_case(corpus_case)

@spec run_case(corpus_case :: map()) :: outcome()

Runs one case through Statifier.Testing.Case.test_scxml/4, or through Mix.Statifier.Corpus.HostCase.run/1 when it carries a host object.

run_paths(paths, root)

@spec run_paths(paths :: [Path.t()], root :: Path.t()) ::
  {:ok, [{Path.t(), outcome()}]} | {:error, String.t()}

Runs the authored cases the regression ratchet names by paths, each path the case's JSON file (Mix.Statifier.Corpus.Authored.case_path/1), relative to root or already joined to it. Returns {path, outcome} pairs in the order given, reading every authored case under root once. A path that names no authored case disagrees, naming itself: the ratchet never passes a case it cannot find. A malformed case tree is refused as Mix.Statifier.Corpus.Authored.read/1 refuses it.

The caller starts the session runtime first (start_runtime/0), as for run/1.

start_runtime()

@spec start_runtime() :: :ok

Starts the :statifier application and places Statifier.Supervisor, the session runtime a case using <send>, <invoke> or a delay runs through (ADR-0027: the library starts no processes of its own). Idempotent: a runtime already placed is left as it is.