Read shapes out of a mix run's captured output.
A mutant's exit code is the primary signal (Mutare.Sandbox.Command decodes
it), but four jobs need to look past the code at the human-readable output:
- Refining a verdict. Exit
1is ambiguous — a genuine harness failure or a mutation that broke the test suite's own compilation — and a BEAM abort can land on any code.suite_compile_error?/1,atom_exhausted?/1,app_start_failure?/1,config_failure?/1,project_failure?/1, andboot_failure?/1are the pure discriminatorsMutare.Sandbox.Command.outcome/2uses to split those cases (see that module's moduledoc for why each is the verdict it is). - Locating a failure.
Mutare.Poisonmaps a failed metamutant compile back to mutant ids (source_location_regex/0+diagnostic_severity/1, and theexpanding macro:frames ofmacro_expansion_stacks/1— whichMutare.Poison.Hintalso reads, for the macro to advise skipping), andMutare.Runner.Baselinenames the tests in a flaky run (test_location_regex/0). - Diagnosing dependencies.
dependency_issue/1distinguishes Mix's dependency-check failures from compile-poisoning so the runner can stop recovery immediately and the Mix task can recommend the correct command in the original project rather than the disposable sandbox. - Summarising a failure. A
:harness_errorhas no verdict to explain it, soMutare.Report.HarnessDiagnosticshows the one output line most likely to:salient_line/1skips mix's routine chatter and prefers a line that heads a failure.
Why these live together
Every pattern here recognises a shape in mix's human-readable output, so they
all break together if mix ever changes its format. They are read for different
jobs by different modules — compile_error_banner/0 to tell a kill from infra,
source_location_regex/0 to map a compile error to a mutant id, test_location_regex/0
to name flaky tests — and the consumers are deliberately not merged (they parse
for different ends). But sourcing every pattern from one module gives a
mix-output-format change a single home.
Everything here is pure, so each discriminator is unit-testable without spawning
mix.
Summary
Types
The remediation class of a Mix dependency-check failure.
One expanding macro: stacktrace frame: the macro's qualified name and arity as the
compiler printed them, and the source location of the frame that follows it — the
macro's call site — or nil when no location frame followed.
Functions
Returns whether output shows the target's Application.start/2 running and failing.
Returns whether output shows the BEAM aborting because the atom table filled.
Returns whether output matches the known boot-failure harness error.
Regex for Mix's == Compilation error in file <path> == banner.
Returns whether output reports an error, exit, or throw while evaluating configuration.
Classify a dependency-check failure in captured Mix output.
Returns the diagnostic block started by a compiler-output line.
The expanding macro: frames in output, one list per stacktrace, innermost first.
Return the last lines lines of captured Mix output.
Returns whether an error, exit, or throw escaped project evaluation.
Returns the captured-output line most likely to explain a failed run, or nil.
Regex for a <file>:<line> source reference in Mix output.
Returns whether output reports a Mix compilation error in a test script.
Regex for a <test_file>:<line> reference in Mix output.
Types
@type dependency_issue() :: :fetch | :compile | :diverged | :unavailable | :invalid
The remediation class of a Mix dependency-check failure.
@type macro_frame() :: %{ name: String.t(), arity: non_neg_integer(), call_site: {String.t(), pos_integer()} | nil }
One expanding macro: stacktrace frame: the macro's qualified name and arity as the
compiler printed them, and the source location of the frame that follows it — the
macro's call site — or nil when no location frame followed.
Functions
Returns whether output shows the target's Application.start/2 running and failing.
This is the signature of a mutation reachable from application startup: mix test
starts the app before it loads test_helper.exs, and the sandbox has already
selected the mutant by then, so such a mutation aborts the boot instead of failing
a test. The baseline boots the same sandbox green, so the runner treats a
persistent one as a kill — see Mutare.Sandbox.Command.outcome/2 and
Mutare.Runner.MutantRun, which first spends the boot-contention retry budget so a
transient startup collision is never charged as a detection.
It matches Mix's Could not start application banner only when the reason names
the start/2 callback — the app exited in it, or it returned an error or a bad
value. A banner that reports a missing application file, an unloadable app or a bad
name is a broken sandbox, not a detection, and returns false.
This predicate only refines an otherwise-:harness_error exit, and boot_failure?/1
takes precedence over it: a node that died mid-boot with its diagnostic erased is
unambiguously infra, whatever Mix managed to print.
Returns whether output shows the BEAM aborting because the atom table filled.
This is the signature of a mutation that mints unbounded atoms. The runner treats that as a kill, like a timeout, because the suite cannot complete with the mutation active.
This predicate only refines an otherwise-:harness_error exit in
Mutare.Sandbox.Command.outcome/2; normal pass, fail, and timeout verdicts
take precedence.
Returns whether output matches the known boot-failure harness error.
The signature is the emulator's terminating during boot message paired with a
secondary :standard_error failure. The original diagnostic is usually gone by
then, and the common cause is resource or connection contention while concurrent
workers start.
The runner still records this as :harness_error, but it can show a more useful
message and use the dedicated boot-failure retry budget. This predicate only
refines an otherwise-:harness_error exit; normal pass, fail, and timeout
verdicts take precedence.
@spec compile_error_banner() :: Regex.t()
Regex for Mix's == Compilation error in file <path> == banner.
The regex captures <path> and is used by suite_compile_error?/1.
Returns whether output reports an error, exit, or throw while evaluating configuration.
Selection precedes configuration, so a mutation called from runtime.exs can
raise before Application.start/2. The sandbox's runtime-config wrapper marks
escaping errors, exits, and throws explicitly, surviving arbitrarily deep library calls.
Config's evaluator frame or a runtime.exs script frame also identifies that
boundary when present. Arbitrary exceptions and
missing config files remain infrastructure failures. The baseline configured
the same sandbox successfully; Mutare.Sandbox.Command.outcome/2 treats this
as :app_start_failure, subject to the same boot-contention retries and
boot_failure?/1 precedence as a failing application callback.
@spec dependency_issue(String.t()) :: dependency_issue() | nil
Classify a dependency-check failure in captured Mix output.
Returns nil for output unrelated to dependencies. The categories deliberately
follow Mix's own recommendations:
:fetch— the lock/source state requiresmix deps.get;:compile— sources exist but requiremix deps.compile;:diverged— dependency declarations conflict;:unavailable— a non-fetchable dependency (normally a local/path dep) is missing from the sandbox's view of the filesystem;:invalid— another status under Mix's unchecked-dependencies banner.
The broad banner proves this is dependency validation, while the narrower
recommendation phrases choose remediation. This keeps an arbitrary compiler
error that merely mentions mix deps.get from being reclassified.
@spec diagnostic_severity(String.t()) :: :error | :warning | nil
Returns the diagnostic block started by a compiler-output line.
The result is :error for an error: header or raised ** (…Error),
:warning for a warning: header, and nil for any other line. Body,
footer, and chatter lines inherit the previous header's severity in the caller.
Mutare.Poison uses this to scan only non-warning lines for mutant locations.
A failed metamutant compile can include warnings caused by mutations, and those
warnings carry the same file:line footer shape as real errors. Separating the
diagnostic blocks keeps warning locations from being treated as poison.
@spec macro_expansion_stacks(String.t()) :: [[macro_frame()]]
The expanding macro: frames in output, one list per stacktrace, innermost first.
The compiler prints a macro-expansion stack innermost-first: the macro whose expansion
raised leads, its enclosing macros follow, and each frame is followed by the source
location that invoked it. Each ** (…) exception header starts a new stack (the lines
before the first header form one too); a stack with no frame is dropped. A location line
before a stack's first frame — the raising macro's own implementation frames — belongs
to no frame and is ignored.
The one reader of this shape: Mutare.Poison.Hint takes each stack's innermost frame as
the macro to advise skipping, and Mutare.Poison takes that frame's call site as the file
to look for its mutants in — so a change to how Elixir prints expansion frames is a single
fix here.
Examples
iex> output =
...> "** (FunctionClauseError) no function clause matching in Size.megabytes/1\n" <>
...> " expanding macro: Size.megabytes/1\n" <>
...> " lib/usage.ex:4: Usage.limit/0\n" <>
...> " (elixir 1.16.0) expanding macro: Kernel.if/2\n" <>
...> " lib/usage.ex:4: Usage.limit/0\n"
iex> Mutare.Sandbox.Command.Output.macro_expansion_stacks(output)
[
[
%{name: "Size.megabytes", arity: 1, call_site: {"lib/usage.ex", 4}},
%{name: "Kernel.if", arity: 2, call_site: {"lib/usage.ex", 4}}
]
]
iex> Mutare.Sandbox.Command.Output.macro_expansion_stacks("** (CompileError) undefined function foo/0\n")
[]
@spec output_tail(String.t(), pos_integer()) :: String.t()
Return the last lines lines of captured Mix output.
Used by baseline, coverage-probe, and Mix-task errors to show enough context without printing an entire suite run.
Returns whether an error, exit, or throw escaped project evaluation.
The sandbox's mix.exs wrapper emits explicit evidence before re-raising,
retaining attribution when a deep library call loses every project stack frame.
An exception alone, or a message merely mentioning mix.exs, is insufficient.
Mutare.Sandbox.Command.outcome/2 applies the same startup-kill policy as for
configuration, including boot-contention retries and boot_failure?/1 precedence.
Returns the captured-output line most likely to explain a failed run, or nil.
Blank lines and routine chatter (compile progress, ExUnit's seed/tag banner,
progress dots, the Finished in footer) are skipped. Of what remains, the first
line that heads a failure wins — a raised ** (…), an error: header, a
dependency-check banner, a failed application start, a boot abort — else the
first remaining line. Lines come back trimmed.
Mutare.Report.HarnessDiagnostic uses it to summarise a :harness_error in one
line; no verdict depends on it.
@spec source_location_regex() :: Regex.t()
Regex for a <file>:<line> source reference in Mix output.
It matches .ex and .exs paths such as lib/foo.ex:5 or
test/foo_test.exs:42, capturing the file and line. Mutare.Poison uses it
to map compile errors back to mutant ids.
Returns whether output reports a Mix compilation error in a test script.
This identifies a mutation that broke test-suite compilation. It matches the
compile_error_banner/0 only when the captured path is a .exs file under a
test/ directory. Lib-file errors and output without the banner return false.
@spec test_location_regex() :: Regex.t()
Regex for a <test_file>:<line> reference in Mix output.
This narrows source_location_regex/0 to _test.exs files. The baseline
runner uses it to name tests involved in a flaky run.