Runs corpus cases through statifier in the calling process's node, the way
a generated test module runs them: each case is one call to
Statifier.Testing.Case.test_scxml/4 with the case's source, description,
initial configuration and steps, so a case agrees exactly when its
generated module would pass.
test_scxml/4 asserts that the active leaf states statifier produces equal
the expected ones after initialization and after every step, so a case
that agrees has configurations that are what statifier produced; a case
that disagrees carries the assertion's own message. Cases run
concurrently, as the generated modules do (async: true); a case needing
a session (<send>, <invoke>, timers) needs the :statifier
application started, which mix statifier.corpus does before it runs any.
A case carrying a host object runs through
Mix.Statifier.Corpus.HostCase instead, which registers the case's send
types with the session it starts and compares the sends handed to them
with the case's expect_sends (ADR-0070 decision 5); when the host
object carries declared_events, it also compares
Statifier.Chart.check_accepts/2's answer with the case's
expect_accepts (ADR-0071 decision 7).
A statifier case whose step carries an expect_position (ADR-0076)
runs through Mix.Statifier.Corpus.HostCase as well, with or without a
host object: once a step's configuration agrees, the session's state
is exported with Statifier.Position.export/1 and compared, in the form
Mix.Statifier.Corpus.PositionExpectation renders, with that step's
expect_position. test_scxml/4 reads no position between its steps,
so a case that states one cannot run there.
Summary
Functions
Runs every case, returning {id, outcome} pairs in the order given.
Runs one case through Statifier.Testing.Case.test_scxml/4, or through
Mix.Statifier.Corpus.HostCase.run/1 when it carries a host object or
a step carries an expect_position; a case with no host object runs
there as a host that registers nothing.
Runs the authored cases the regression ratchet names by paths, each
path the case's JSON file (Mix.Statifier.Corpus.Authored.case_path/1),
relative to root or already joined to it. Returns {path, outcome}
pairs in the order given, reading every authored case under root
once. A path that names no authored case disagrees, naming itself: the
ratchet never passes a case it cannot find. A malformed case tree is
refused as Mix.Statifier.Corpus.Authored.read/1 refuses it.
Starts the :statifier application and places Statifier.Supervisor,
the session runtime a case using <send>, <invoke> or a delay runs
through (ADR-0027: the library starts no processes of its own). Idempotent:
a runtime already placed is left as it is.
Types
@type outcome() :: :agree | {:disagree, String.t()}
What running one case found.
Functions
Runs every case, returning {id, outcome} pairs in the order given.
Runs one case through Statifier.Testing.Case.test_scxml/4, or through
Mix.Statifier.Corpus.HostCase.run/1 when it carries a host object or
a step carries an expect_position; a case with no host object runs
there as a host that registers nothing.
@spec run_paths(paths :: [Path.t()], root :: Path.t()) :: {:ok, [{Path.t(), outcome()}]} | {:error, String.t()}
Runs the authored cases the regression ratchet names by paths, each
path the case's JSON file (Mix.Statifier.Corpus.Authored.case_path/1),
relative to root or already joined to it. Returns {path, outcome}
pairs in the order given, reading every authored case under root
once. A path that names no authored case disagrees, naming itself: the
ratchet never passes a case it cannot find. A malformed case tree is
refused as Mix.Statifier.Corpus.Authored.read/1 refuses it.
The caller starts the session runtime first (start_runtime/0), as for
run/1.
@spec start_runtime() :: :ok
Starts the :statifier application and places Statifier.Supervisor,
the session runtime a case using <send>, <invoke> or a delay runs
through (ADR-0027: the library starts no processes of its own). Idempotent:
a runtime already placed is left as it is.
It also puts OTP's :inets, :public_key and :ssl on the code path,
in that order. Mix leaves them off for an application no dependency
lists, and they are needed: a case whose host runs an Event I/O
Processor delivers through the processor's default transport on
:httpc and a loopback front on :inets httpd
(Mix.Statifier.Corpus.HostCase), and :httpc reads :public_key
even for a plain http: request. It stops at the first
application it cannot find and raises Mix.Error naming that one alone,
which may be an application one of the three requires (each one's
requirements are looked up before the next of the three), so a host
missing more than one learns of them one at a time.