Mutare.Coverage (mutare v0.3.1)

Copy Markdown View Source

Read back the coverage the metamutant recorded during the probe run.

The metamutant self-records coverage synchronously, in the test process, as the suite runs (Mutare.Coverage.Recorder generates the recording code); at suite end an ExUnit.after_suite/1 hook dumps it to a file. This module reads that dump.

The dump is three things, all keyed by mutant id (no metamutant↔original line mapping — the original line is only for the report):

  • :aggregate — the set of mutant ids whose selector ran at all, in any process (test body, setup, setup_all, a spawned task). This is the no-coverage signal: an id not in it can never be killed, so skip it and keep it out of the score's denominator.
  • :by_file — %{test_file => MapSet(mutant ids)}: which mutant ids each test file covered. A line running in the test process is labeled directly; one running in a Task it spawned is attributed via the caller chain; one running in a module's setup_all is attributed via the __ex_unit__/2 stacktrace frame (module-granular, the file selection needs). This drives per-file test selection.
  • :unlabeled — the set of mutant ids whose selector ran in a process with no recoverable test label at all — a bare spawn, a setup-registered on_exit closure, or the rare setup_all whose work happened off-stack in a Task it spawned (an on_exit registered in a test body is recovered, via its closure frame in ExUnit's per-test runner process). An id here was covered, but cannot be attributed to a specific test, so the caller runs the whole suite for it — even if :by_file also attributes it to some file, since that partial attribution would otherwise mask the unlabeled coverage and manufacture a false survivor.

Two more keys carry the finer test-case granularity :tests selection uses (both keyed by mutant id):

  • :by_test — %{mutant id => MapSet(runnable test names)}: the individual ExUnit tests (test/doctest/property names) that covered each id, so a mutant can run mix test <file> --only test:<name> instead of the whole file.
  • :wholefile — the set of ids with a labeled but non-narrowable attribution (a setup_all or an on_exit, which cover through a module-scoped context, not a single runnable test). :tests must run the whole file for such an id — narrowing to named tests would drop the covering context and manufacture a false survivor.

Mutare.Runner.CoverageProbe reconciles them all. A valid empty aggregate means no emitted mutant was covered. The helper writes an explicit error if any capture table is missing, so lost capture data cannot masquerade as zero hits, and it always writes all five keys — a dump missing one is malformed, not a partial capture, and is rejected like any other wrong shape.

Schema dumps record {file_namespace, local_id} identities; standalone transforms record integers. On disk, each id collection groups local ids under their namespace (nil for integers), so a file path appears once per group, not once per id. read_dump/2 flattens the groups and accepts Mutare.RuntimeId.index(schema.sites) to translate every collection to report ids before the runner consumes it. An unknown identity invalidates the dump and triggers run-all, never false no-coverage. read_dump/1 exposes the runtime identities as recorded.

Why not :cover: its counters live in a single global table keyed {module, line} with no per-process partition, so attributing coverage to a test in one run needs a per-test snapshot, and the only global per-test signal ExUnit emits is an async cast to formatters that races test execution (fast async: false modules' coverage is lost). Self-recording in the metamutant sidesteps that entirely — and is process-agnostic for the aggregate, so it is a better no-coverage detector than :cover ever was.

Summary

Types

t()

The decoded dump. Keyed by mutant id throughout

Functions

Read and decode the probe's coverage dump at path.

Types

t()

@type t() :: %Mutare.Coverage{
  aggregate: MapSet.t(Mutare.RuntimeId.t()),
  by_file: %{required(String.t()) => MapSet.t(Mutare.RuntimeId.t())},
  by_test: %{required(Mutare.RuntimeId.t()) => MapSet.t(String.t())},
  unlabeled: MapSet.t(Mutare.RuntimeId.t()),
  wholefile: MapSet.t(Mutare.RuntimeId.t())
}

The decoded dump. Keyed by mutant id throughout:

  • aggregate — the process-agnostic hit set (no-coverage detection).
  • by_file — per-test-file attribution (:coverage selection, and the :tests fallback).
  • unlabeled — the whole-suite hit set.
  • by_test — per-test-case attribution: id => runnable test names that covered it (:tests narrowing).
  • wholefile — ids with a labeled but non-narrowable attribution (setup_all/on_exit), which :tests must not narrow to named tests.

Functions

read_dump(path, report_ids \\ nil)

@spec read_dump(Path.t(), map() | nil) :: {:ok, t()} | {:error, term()}

Read and decode the probe's coverage dump at path.

An optional runtime-to-report index translates every recorded identity. Omitting it returns raw runtime identities, useful for inspecting a standalone dump.

Returns {:error, _} on anything unusable (missing file, truncated/garbled payload, a capture error from the helper, unexpected shape — including a well-formed outer map whose nested keys, ids or collections are the wrong type) — the caller degrades such uncertainty to running the whole suite, never to a false :no_coverage. It raises for no input.