Runs a project's own check as a stage, described declaratively in .quality.exs.

This is the ergonomic half of custom stages: a house rule, a schema linter, a mix task or a shell script gate becomes a stage of a mix quality run without anyone writing a module. The other half is a module implementing the ExQuality.Stage contract, for anything the command form cannot express.

custom: [
  [
    key: :nullability,
    name: "Nullability",
    command: "mix",
    args: ["schema.nullability", "--format", "json"],
    env: [{"MIX_ENV", "test"}],
    kind: :reader
  ]
]

Exit code 0 is :ok, anything else is :error. The command is run with stderr_to_stdout: true, as every other shelling stage is, so a tool that writes its complaint to stderr is not thrown away.

Naming the command

A bare name is looked up on the PATH. A command containing / is a path, and is expanded before it runs, so a project's own script can be named directly:

command: "bin/checks/schema.sh"

The path is relative to cd: when one is given and to the project root otherwise, so an entry reads as the shell it looks like: cd <cd> && <command> <args>. An absolute path is used as it stands.

The finding contract

A command that wants structured findings prints one JSON document on stdout:

{
  "summary": "2 unsound claims",
  "stats": {"finding_count": 2},
  "findings": [
    {
      "file": "lib/contacts/contact.ex",
      "line": 14,
      "column": null,
      "app": "web",
      "severity": "error",
      "check": "unsound",
      "message": "field :email is typed non-nil but the column is nullable"
    }
  ]
}

Only file and message are required per finding. app may be omitted and is inferred from the path. See ExQuality.Finding.from_map/2.

Anything that does not parse falls through to output verbatim, which is the rule the printer and the report already follow. parse: :none skips the attempt for a command known to print prose, so a tool that happens to emit JSON for some other reason is not misread.

Not applicable

A custom check often has a prerequisite ExQuality cannot know about: a migrated test database, a running service, a generated file. Without a way to say "not applicable" the stage fails with an error that reads like a code problem. skip_exit_code: 2 lets the command exit 2 and have the stage report :skipped with its own reason, which keeps the invariant that a stage saying nothing would read as a stage that passed.

The reason is the document's summary when the command wrote one, and the first line of output otherwise. Prefer the document: a first line is hostage to whatever the toolchain prints ahead of the command's own output, and mix in particular emits ==> app headers for an umbrella and a build-lock notice when another stage holds the lock.

Reader versus writer

kind: :reader is the default, and it is what most custom checks are: they read source, or query a database. A command that compiles, generates, or writes anything under _build or the repository must declare kind: :writer, because the analysis phase runs its readers concurrently and a stage that rewrites the beams underneath them makes another stage report a failure about the build rather than about the code.

MIX_ENV=test mix <task> is normally still a reader here, because the Compile stage has already built dev and test before the analysis phase starts. That is the most common shape a custom command takes and it looks like a writer, so it is worth saying.