# Stages What each stage runs, what it reports, and where its threshold comes from. A run has three phases. **Format** fixes what it can. **Compile** gates the rest: analysing code that does not build wastes the wait. Everything else runs in parallel and prints as it finishes. ## Format Runs `mix format`, having first checked with `--check-formatted` so it can report how many files it changed. ``` ✓ Format: Formatted 3 files (0.2s) ✓ Format: No changes needed (458ms) ``` A project with no `.formatter.exs` is reported as skipped rather than failed: it never had the config file, so there is nothing to enforce. A `mix format` that fails - a syntax error, say - is reported as a failure, not as a green tick over a file that could not be parsed. ## Compile Compiles dev and test in parallel, with `--warnings-as-errors` by default. ``` ✓ Compile: dev + test compiled (warnings as errors) (325ms) ✗ Compile: dev compilation failed ``` A failure here stops the run. The analysis stages that were never reached are reported as skipped, with `compile failed` as the reason, so they do not read as stages that passed. Set `compile: [warnings_as_errors: false]` to allow warnings. ## Credo Runs `mix credo --format json`, so an issue is never dropped for having an unfamiliar line shape. ``` ✓ Credo: No issues (775ms) ✗ Credo: 5 issues (2 readability, 3 design) (0.4s) ``` Each issue becomes a finding naming the check that produced it: ``` lib/user.ex 42:3 [info] Modules should have a @moduledoc tag. (Credo.Check.Readability.ModuleDoc) ``` `credo: [strict: true]` is the default. Configure credo itself in `.credo.exs`. ### More than one config A `.credo.exs` can declare several configs, and a check that lives in a second one does not run under the default. That happens when a set of files sits outside credo's default `files.included` - migrations are the usual case - because folding them into the default config would subject them to the whole default check set. ```elixir credo: [configs: ["default", "migrations"]] ``` Each name is one `mix credo` invocation, in the order given, and the results merge into a single stage result. Findings are deduplicated on file, line, column and check, so two configs with overlapping `files` globs do not double a count. A run that fails without reporting issues names the config it came from, because the likeliest cause is a name `.credo.exs` does not define: ``` ✗ Credo: Check failed in "migrations" (see output) ``` The runs are serial. `mix credo` can compile, and two concurrent mix runs against the same `_build` is exactly what the writer/reader split exists to prevent. The stage as a whole still runs in parallel with the other analysis stages. Without `configs`, one run happens with no `--config-name`, which is credo's own default. ## Dialyzer Runs `mix dialyzer --no-compile --format short --format dialyxir`. The short form of each warning becomes a finding naming the warning (`no_return`, `pattern_match`); dialyxir's long explanation is kept alongside it, in the stage's output and in the JSON report. ``` ✓ Dialyzer: No warnings (32.1s) ✗ Dialyzer: 2 warnings (12.4s) ``` Dialyzer analyses against a PLT, a cache of every module it has already seen. Building one takes minutes, analysing against a warm one takes seconds. A run that has to build it says so while it happens, rather than looking hung: ``` ⋯ Dialyzer: building PLT (this is a one-time cost) ✓ Dialyzer: No warnings (PLT built this run) (252.4s) ``` `mix quality.plt` builds it outside a run, so a container image or CI job can cache it. See [ci.md](ci.md). `--quick` skips this stage. ## Dependencies Two checks in one stage. `mix deps.unlock --check-unused` always; and `mix deps.audit --format json` when `:mix_audit` is installed. ``` ✓ Dependencies: No unused dependencies (0.3s) ✗ Dependencies: 1 vulnerability (1 moderate) (2.5s) ``` Each vulnerability becomes a finding against the lockfile, naming the advisory, the version in use and the version that fixes it. Unused dependencies become findings too. ``` mix.lock - [error] decimal 2.3.0: Unbounded exponent in `Decimal.new` enables unauthenticated DoS (moderate severity, patched in 3.0.0) (GHSA-rhv4-8758-jx7v) ``` ## Doctor Runs `mix doctor`, which enforces the documentation coverage thresholds in the project's `.doctor.exs`. ``` ✓ Doctor: Passed (519ms) ✗ Doctor: Documentation coverage below threshold ``` `doctor: [summary_only: true]` prints only the summary. ## Gettext Reads the project's `.po` files for untranslated and fuzzy entries, under every umbrella child app as well as the root, and reports each one at its `msgid`. ``` ✓ Gettext: All translations complete (12 files) (30ms) ✗ Gettext: 4 missing, 2 fuzzy translations ○ Gettext: skipped (no .po files found) ``` Files in the source locale are not checked, because the source locale is untranslated by definition. It is `"en"` unless `gettext: [source_locale: ...]` says otherwise, and `errors.po` is excluded for the same reason. A run left with nothing to read reports itself as skipped rather than complete. This stage does not run `mix gettext.extract --merge`. That task writes: it rewrites `.pot` and `.po` files, and it compiles the project to do it, which changes the build the other stages are reading. `gettext: [extract: true]` opts back in, and the run then serialises the stage rather than running it alongside the readers. No other stage can write to your repository. ## Custom stages A project's own check runs as a stage rather than beside it, declared under `custom:` in `.quality.exs`. It prints, times and reports like a built-in one: ``` ✓ Nullability: No unsound claims (1.1s) ✗ Nullability: 2 unsound claims (0.9s) ○ Nullability: skipped (--skip nullability) ``` The entry forms, the option table and the validation rules are in [configuration.md](configuration.md#custom-stages). Two things belong here. ### The reader/writer declaration is load-bearing The analysis phase runs its stages concurrently, and that is only safe while every one of them is a reader. A command that recompiles the project rewrites the beams the other stages are part-way through reading, and the stage that notices reports a failure about the build rather than about the code. `kind: :reader` is the default, because most custom checks read source or query a database. **If your command compiles, generates, or writes anything under `_build` or the repository, declare `kind: :writer`.** A writer runs on its own, before the readers, the same way the Compile stage is a serialized gate. `MIX_ENV=test mix ` is normally still a reader here, because the Compile stage has already built dev and test before the analysis phase starts. That is the most common shape a custom command takes and it looks like a writer, which is why it is worth saying. ### The JSON finding contract A command that wants per-finding routing rather than a wall of text prints one JSON document on stdout: ```json { "summary": "2 unsound claims", "stats": {"finding_count": 2}, "findings": [ { "file": "apps/walt_ui/lib/walt_ui/contacts/contact.ex", "line": 14, "column": null, "app": "walt_ui", "severity": "error", "check": "unsound", "message": "field :email is typed non-nil but the column is nullable" } ] } ``` Only `file` and `message` are required per finding. `app` may be omitted and is inferred from the path. `severity` is `error`, `warning` or `info`. `summary` and `stats` are optional; without them the stage counts its own findings. Anything that does not parse falls through to `output` verbatim, which is the rule the printer and the report already follow everywhere else. `parse: :none` skips the attempt for a command known to print prose, so a tool that happens to emit JSON for some other reason is not misread. ### Not applicable A custom check often has a prerequisite ExQuality cannot know about: a migrated test database, a running service, a generated file. Without a way to say "not applicable" the stage fails with an error that reads like a code problem. `skip_exit_code: 2` lets the command exit 2 and have the stage report itself as skipped, with the command's own first line as the reason: ``` ○ Nullability: skipped (test database is not migrated) ``` Which keeps the invariant a run depends on: a stage that says nothing would read as a stage that passed. ## Aliased tasks ExQuality shells out to the real `mix credo`, `mix dialyzer`, `mix format`, `mix sobelow`, `mix deps.unlock` and `mix test.coverage`, and reads what they print. Mix resolves aliases before tasks, so a project that defines an alias with one of those names silently changes what the stage measures. A stage checks before shelling out, and refuses rather than reporting a number about a command it did not issue: ``` ✗ Format: mix format is aliased in mix.exs ✗ Sobelow: mix sobelow is aliased in mix.exs ``` Rename the alias (`sobelow.all` is the usual choice) and point your own scripts at the new name. `mix test` is the deliberate exception: a `test:` alias that runs migrations first is near-universal, and running it is what the suite needs. ## Sobelow Runs `mix sobelow` on a Phoenix project. Sobelow reports findings at three confidence levels, but only those at or above the project's `exit:` threshold block a build. ExQuality renders those and reports the rest as a count: ``` ✗ Sobelow: 2 blocking findings (1 high, 1 medium), 3 informational not shown ``` The threshold comes from `.sobelow-conf` when that file sets one, because what blocks a build is a security decision that belongs with the security config. `sobelow: [exit: "high"]` in `.quality.exs` only supplies a default for a project without one; the default when neither says is `"medium"`. Pass `--verbose`, or set `sobelow: [show_informational: true]`, to render the findings below the threshold as well. ExQuality will never suggest editing `.sobelow-conf` to make a run pass. A tool that silences its own findings to go green is a regression dressed as a pass. ## Tests Runs `mix test`, or `mix coveralls` when coverage is being measured. ``` ✓ Tests: 345 of 345 passed (4.1s) ✓ Tests: 248 passed, 87.3% coverage (5.2s) ✗ Tests: 3 of 4,180 failed (web: 3) ``` Extra arguments reach the test command via `--` or `test: [args: [...]]`. See [configuration.md](configuration.md). ## Coverage Coverage is part of the Tests stage, and `--quick` turns it off. **The threshold is not configured in ExQuality.** It is read from whichever coverage tool the project already uses, so there is one source of truth: | Tool | Threshold read from | |---|---| | `:excoveralls` (`mix coveralls`) | `coveralls.json` → `coverage_options.minimum_coverage`, or `mix.exs` → `test_coverage: [minimum_coverage: 80.0]` | | Elixir's own (`mix test --cover`) | `mix.exs` → `test_coverage: [summary: [threshold: 90]]` | Without excoveralls, coverage is measured only when the project states a threshold that way. Elixir applies a default of 90% whether or not a project has ever thought about coverage, and ExQuality will not turn a green run red over a number nobody chose. To measure anyway: ```elixir # .quality.exs test: [coverage: true] # or false to never measure ``` When the threshold is missed, the modules under it are reported as findings, rather than the whole per-module table: ``` ✗ Tests: Coverage 62.5% (required: 90.0%) lib/my_app/mailer.ex - [error] MyApp.Mailer is 0.0% covered (threshold 90.0%) lib/my_app/thing.ex - [error] MyApp.Thing is 33.3% covered (threshold 90.0%) ``` Lowering the threshold to make a run pass is a decision for a human, not a fix.