Runs the test suite with optional coverage analysis.

• Uses mix coveralls if :excoveralls is in deps
• Uses mix test --cover if it is not, and the project asks for coverage
• Uses mix test otherwise, and always in quick mode
• Parses test count, pass/fail counts
• Parses coverage percentage if available
• Reads the coverage threshold from the tool's own config (single source of truth), whichever of the two tools is in use

In quick mode: runs mix test (tests must pass, but coverage threshold is not enforced).

Native coverage

Elixir has shipped coverage since 1.11, so a project without :excoveralls is not a project without coverage. mix test --cover is used for those, reading test_coverage: [summary: [threshold: N]] from mix.exs the same way the built-in tool does.

It is used only when the project sets that threshold, or asks for coverage explicitly with test: [coverage: true] in .quality.exs. Elixir's built-in threshold defaults to 90% whether or not a project has ever thought about coverage, and turning a green run red over a number nobody chose is not a gate anyone asked for. A stated threshold is the project saying it wants one.

Umbrellas

An umbrella mix test prints one summary line per app, so every summary is read and summed rather than only the first. Reporting the first app's numbers as the suite's is worse than reporting none: a confident, specific, false count ("3 of 12 failed" when the truth is "3 of 4,180 failed") reads as more trustworthy than no count at all.

Apps with failures are named in the summary. Apps that passed are not, since the failing ones are what a reader acts on.

Native coverage in an umbrella is measured in two commands, because a per-app mix test --cover only sees its own app's modules: a module exercised by another app's tests reads as 0%. So the run exports instead (--export-coverage), and mix test.coverage aggregates the exports into one table and one total.

Findings

A failing test is the most common thing a run has to hand off, so each one is parsed into a finding carrying the file, the line the test is defined at, the app, the test module and the assertion message. The alternative is 30 KB of run log with one failure buried in it.

The failure block's own stacktrace: lines are relative to the app, while the header line under the test's name is relative to the umbrella root, so the header line is what is read: a path that does not open from where the report was generated cannot be routed anywhere.

The parse is used only when it accounts for every failure the suite counted. Three findings for four failures would read as the complete list, which is worse than showing the log; a short parse falls back to output in full.

A failing coverage check reports the modules under the threshold, not the whole table. A 400-module umbrella prints 400 rows; the handful below the line are the ones a reader acts on. Modules are reported only when the run actually failed on coverage, because the threshold applies to the total: a passing run can hold a low module and there is nothing to do about it.