Runs the test suite with optional coverage analysis.

• Uses mix coveralls if :excoveralls is in deps
• Uses mix test --cover if it is not, and the project asks for coverage
• Uses mix test otherwise, and always in quick mode
• Parses test count, pass/fail counts
• Parses coverage percentage if available
• Reads the coverage threshold from the tool's own config (single source of truth), whichever of the two tools is in use

In quick mode: runs mix test (tests must pass, but coverage threshold is not enforced).

Native coverage

Elixir has shipped coverage since 1.11, so a project without :excoveralls is not a project without coverage. mix test --cover is used for those, reading test_coverage: [summary: [threshold: N]] from mix.exs the same way the built-in tool does.

It is used only when the project sets that threshold, or asks for coverage explicitly with test: [coverage: true] in .quality.exs. Elixir's built-in threshold defaults to 90% whether or not a project has ever thought about coverage, and turning a green run red over a number nobody chose is not a gate anyone asked for. A stated threshold is the project saying it wants one.

Umbrellas

An umbrella mix test prints one summary line per app, so every summary is read and summed rather than only the first. Reporting the first app's numbers as the suite's is worse than reporting none: a confident, specific, false count ("3 of 12 failed" when the truth is "3 of 4,180 failed") reads as more trustworthy than no count at all.

Apps with failures are named in the summary. Apps that passed are not, since the failing ones are what a reader acts on.

Native coverage in an umbrella is measured in two commands, because a per-app mix test --cover only sees its own app's modules: a module exercised by another app's tests reads as 0%. So the run exports instead (--export-coverage), and mix test.coverage aggregates the exports into one table and one total.

Findings

A failing test is the most common thing a run has to hand off, so each one is parsed into a finding carrying the file, the line the test is defined at, the app, the test module and the assertion message. The alternative is 30 KB of run log with one failure buried in it.

The failure block's own stacktrace: lines are relative to the app, while the header line under the test's name is relative to the umbrella root, so the header line is what is read: a path that does not open from where the report was generated cannot be routed anywhere.

The parse is used only when it accounts for every failure the suite counted. Three findings for four failures would read as the complete list, which is worse than showing the log; a short parse falls back to output in full.

A failing coverage check reports the modules under the threshold, not the whole table. A 400-module umbrella prints 400 rows; the handful below the line are the ones a reader acts on. Modules are reported only when the run actually failed on coverage, because the threshold applies to the total: a passing run can hold a low module and there is nothing to do about it.

Scoped runs

test: [scope: :changed], or --test-scope changed, runs only the test files covering the code that changed. This is the whole reason an agent can afford to run checks between edits: the suite is most of a run's wall clock, and a one-file change needs a handful of files rather than all of them. See ExQuality.Scope, which owns resolving the scope and the rule that an empty resolution runs the full suite rather than reporting a green run of nothing.

Coverage in a scoped run is not lower, it is absent. It is reported as skipped with a reason and never as a number, because a percentage measured over a subset of the suite is not a smaller truth, it is a different and misleading one, and anything downstream that ratchets a recorded figure would lower it against a run that never measured.

The result carries what the run actually covered under meta, so a "status": "ok" in the report can be interpreted rather than assumed to be a full green.