Mutare.Sandbox.Command.Output (mutare v0.4.1)

Copy Markdown View Source

Read shapes out of a mix run's captured output.

A mutant's exit code is the primary signal (Mutare.Sandbox.Command decodes it), but four jobs need to look past the code at the human-readable output:

Why these live together

Every pattern here recognises a shape in mix's human-readable output, so they all break together if mix ever changes its format. They are read for different jobs by different modules — compile_error_banner/0 to tell a kill from infra, source_location_regex/0 to map a compile error to a mutant id, test_location_regex/0 to name flaky tests — and the consumers are deliberately not merged (they parse for different ends). But sourcing every pattern from one module gives a mix-output-format change a single home.

Everything here is pure, so each discriminator is unit-testable without spawning mix.

Summary

Types

The remediation class of a Mix dependency-check failure.

One expanding macro: stacktrace frame: the macro's qualified name and arity as the compiler printed them, and the source location of the frame that follows it — the macro's call site — or nil when no location frame followed.

Functions

Returns whether output shows the target's Application.start/2 running and failing.

Returns whether output shows the BEAM aborting because the atom table filled.

Returns whether output matches the known boot-failure harness error.

Regex for Mix's == Compilation error in file <path> == banner.

Returns whether output reports an error, exit, or throw while evaluating configuration.

Classify a dependency-check failure in captured Mix output.

Returns the diagnostic block started by a compiler-output line.

The expanding macro: frames in output, one list per stacktrace, innermost first.

Return the last lines lines of captured Mix output.

Returns whether an error, exit, or throw escaped project evaluation.

Returns the captured-output line most likely to explain a failed run, or nil.

Regex for a <file>:<line> source reference in Mix output.

Returns whether output reports a Mix compilation error in a test script.

Regex for a <test_file>:<line> reference in Mix output.

Types

dependency_issue()

@type dependency_issue() :: :fetch | :compile | :diverged | :unavailable | :invalid

The remediation class of a Mix dependency-check failure.

macro_frame()

@type macro_frame() :: %{
  name: String.t(),
  arity: non_neg_integer(),
  call_site: {String.t(), pos_integer()} | nil
}

One expanding macro: stacktrace frame: the macro's qualified name and arity as the compiler printed them, and the source location of the frame that follows it — the macro's call site — or nil when no location frame followed.

Functions

app_start_failure?(output)

@spec app_start_failure?(String.t()) :: boolean()

Returns whether output shows the target's Application.start/2 running and failing.

This is the signature of a mutation reachable from application startup: mix test starts the app before it loads test_helper.exs, and the sandbox has already selected the mutant by then, so such a mutation aborts the boot instead of failing a test. The baseline boots the same sandbox green, so the runner treats a persistent one as a kill — see Mutare.Sandbox.Command.outcome/2 and Mutare.Runner.MutantRun, which first spends the boot-contention retry budget so a transient startup collision is never charged as a detection.

It matches Mix's Could not start application banner only when the reason names the start/2 callback — the app exited in it, or it returned an error or a bad value. A banner that reports a missing application file, an unloadable app or a bad name is a broken sandbox, not a detection, and returns false.

This predicate only refines an otherwise-:harness_error exit, and boot_failure?/1 takes precedence over it: a node that died mid-boot with its diagnostic erased is unambiguously infra, whatever Mix managed to print.

atom_exhausted?(output)

@spec atom_exhausted?(String.t()) :: boolean()

Returns whether output shows the BEAM aborting because the atom table filled.

This is the signature of a mutation that mints unbounded atoms. The runner treats that as a kill, like a timeout, because the suite cannot complete with the mutation active.

This predicate only refines an otherwise-:harness_error exit in Mutare.Sandbox.Command.outcome/2; normal pass, fail, and timeout verdicts take precedence.

boot_failure?(output)

@spec boot_failure?(String.t()) :: boolean()

Returns whether output matches the known boot-failure harness error.

The signature is the emulator's terminating during boot message paired with a secondary :standard_error failure. The original diagnostic is usually gone by then, and the common cause is resource or connection contention while concurrent workers start.

The runner still records this as :harness_error, but it can show a more useful message and use the dedicated boot-failure retry budget. This predicate only refines an otherwise-:harness_error exit; normal pass, fail, and timeout verdicts take precedence.

compile_error_banner()

@spec compile_error_banner() :: Regex.t()

Regex for Mix's == Compilation error in file <path> == banner.

The regex captures <path> and is used by suite_compile_error?/1.

config_failure?(output)

@spec config_failure?(String.t()) :: boolean()

Returns whether output reports an error, exit, or throw while evaluating configuration.

Selection precedes configuration, so a mutation called from runtime.exs can raise before Application.start/2. The sandbox's runtime-config wrapper marks escaping errors, exits, and throws explicitly, surviving arbitrarily deep library calls. Config's evaluator frame or a runtime.exs script frame also identifies that boundary when present. Arbitrary exceptions and missing config files remain infrastructure failures. The baseline configured the same sandbox successfully; Mutare.Sandbox.Command.outcome/2 treats this as :app_start_failure, subject to the same boot-contention retries and boot_failure?/1 precedence as a failing application callback.

dependency_issue(output)

@spec dependency_issue(String.t()) :: dependency_issue() | nil

Classify a dependency-check failure in captured Mix output.

Returns nil for output unrelated to dependencies. The categories deliberately follow Mix's own recommendations:

  • :fetch — the lock/source state requires mix deps.get;
  • :compile — sources exist but require mix deps.compile;
  • :diverged — dependency declarations conflict;
  • :unavailable — a non-fetchable dependency (normally a local/path dep) is missing from the sandbox's view of the filesystem;
  • :invalid — another status under Mix's unchecked-dependencies banner.

The broad banner proves this is dependency validation, while the narrower recommendation phrases choose remediation. This keeps an arbitrary compiler error that merely mentions mix deps.get from being reclassified.

diagnostic_severity(line)

@spec diagnostic_severity(String.t()) :: :error | :warning | nil

Returns the diagnostic block started by a compiler-output line.

The result is :error for an error: header or raised ** (…Error), :warning for a warning: header, and nil for any other line. Body, footer, and chatter lines inherit the previous header's severity in the caller.

Mutare.Poison uses this to scan only non-warning lines for mutant locations. A failed metamutant compile can include warnings caused by mutations, and those warnings carry the same file:line footer shape as real errors. Separating the diagnostic blocks keeps warning locations from being treated as poison.

macro_expansion_stacks(output)

@spec macro_expansion_stacks(String.t()) :: [[macro_frame()]]

The expanding macro: frames in output, one list per stacktrace, innermost first.

The compiler prints a macro-expansion stack innermost-first: the macro whose expansion raised leads, its enclosing macros follow, and each frame is followed by the source location that invoked it. Each ** (…) exception header starts a new stack (the lines before the first header form one too); a stack with no frame is dropped. A location line before a stack's first frame — the raising macro's own implementation frames — belongs to no frame and is ignored.

The one reader of this shape: Mutare.Poison.Hint takes each stack's innermost frame as the macro to advise skipping, and Mutare.Poison takes that frame's call site as the file to look for its mutants in — so a change to how Elixir prints expansion frames is a single fix here.

Examples

iex> output =
...>   "** (FunctionClauseError) no function clause matching in Size.megabytes/1\n" <>
...>     "    expanding macro: Size.megabytes/1\n" <>
...>     "    lib/usage.ex:4: Usage.limit/0\n" <>
...>     "    (elixir 1.16.0) expanding macro: Kernel.if/2\n" <>
...>     "    lib/usage.ex:4: Usage.limit/0\n"
iex> Mutare.Sandbox.Command.Output.macro_expansion_stacks(output)
[
  [
    %{name: "Size.megabytes", arity: 1, call_site: {"lib/usage.ex", 4}},
    %{name: "Kernel.if", arity: 2, call_site: {"lib/usage.ex", 4}}
  ]
]
iex> Mutare.Sandbox.Command.Output.macro_expansion_stacks("** (CompileError) undefined function foo/0\n")
[]

output_tail(output, lines \\ 20)

@spec output_tail(String.t(), pos_integer()) :: String.t()

Return the last lines lines of captured Mix output.

Used by baseline, coverage-probe, and Mix-task errors to show enough context without printing an entire suite run.

project_failure?(output)

@spec project_failure?(String.t()) :: boolean()

Returns whether an error, exit, or throw escaped project evaluation.

The sandbox's mix.exs wrapper emits explicit evidence before re-raising, retaining attribution when a deep library call loses every project stack frame. An exception alone, or a message merely mentioning mix.exs, is insufficient. Mutare.Sandbox.Command.outcome/2 applies the same startup-kill policy as for configuration, including boot-contention retries and boot_failure?/1 precedence.

salient_line(output)

@spec salient_line(String.t()) :: String.t() | nil

Returns the captured-output line most likely to explain a failed run, or nil.

Blank lines and routine chatter (compile progress, ExUnit's seed/tag banner, progress dots, the Finished in footer) are skipped. Of what remains, the first line that heads a failure wins — a raised ** (…), an error: header, a dependency-check banner, a failed application start, a boot abort — else the first remaining line. Lines come back trimmed.

Mutare.Report.HarnessDiagnostic uses it to summarise a :harness_error in one line; no verdict depends on it.

source_location_regex()

@spec source_location_regex() :: Regex.t()

Regex for a <file>:<line> source reference in Mix output.

It matches .ex and .exs paths such as lib/foo.ex:5 or test/foo_test.exs:42, capturing the file and line. Mutare.Poison uses it to map compile errors back to mutant ids.

suite_compile_error?(output)

@spec suite_compile_error?(String.t()) :: boolean()

Returns whether output reports a Mix compilation error in a test script.

This identifies a mutation that broke test-suite compilation. It matches the compile_error_banner/0 only when the captured path is a .exs file under a test/ directory. Lib-file errors and output without the banner return false.

test_location_regex()

@spec test_location_regex() :: Regex.t()

Regex for a <test_file>:<line> reference in Mix output.

This narrows source_location_regex/0 to _test.exs files. The baseline runner uses it to name tests involved in a flaky run.