Mix.Statifier.AdrJudge (Statifier v2.2.0)

Judges the current branch's diff against a registry of judged ADRs, each carrying its own path scope, ADR text, and failure vocabulary, using two independent model calls per ADR: one proposes violations, a second is prompted to refute each one. Only a proposed violation the refute pass fails to overturn becomes a finding - a single pass reporting whatever it first notices is exactly what the adversarial-verification requirement on this check rules out, because a false positive here blocks a commit (CLAUDE.md: "never go green by weakening the check" means the fix for a bad finding has to be "the check was wrong," not "disable the check" - so the bar to reach gate-failure status is higher than an FYI).

The refute pass now sees the same diff hunks the propose pass saw, not just the ADR text and the one-sentence claim - the same material the propose pass reasoned from, so "the change does not show X" is checkable by the refute pass rather than an assumption it cannot verify - and its prompt requires the refuting argument to be grounded in that material - the ADR text, the claim, or the hunks. A defence of the shape "the change does not show X, but X might exist elsewhere" is unverifiable with no tool access to check it, so it does not overturn a claim (the refute pass had previously overturned a real ADR-0012 violation on exactly that hypothesis, a mechanism the diff never showed and the pass had no way to confirm). The tie rule survives narrowed to match: ambiguity within the shown material still breaks toward "not a violation," but uncertainty about material never shown is not a tie.

@default_model is measured, not assumed: the fixture corpus (test/mix/statifier/adr_judge_corpus_test.exs, mix test --only adr_judge_corpus) ran against both claude-haiku-4-5-20251001 and claude-sonnet-5. Both scored 0 false negatives and 0 false positives out of 8 fixtures - grounding the refute prompt in the diff hunks was what fixed the bug, and neither model has an accuracy edge over the other on this corpus. Accuracy therefore does not pick the default, and latency does: sonnet ran the corpus in 91.4s against haiku's 272.4s, and a real three-entry mix adr.judge in 19.6s against haiku's 54.4s. The default is claude-sonnet-5 for that wall-clock margin, a deliberate trade of token cost for gate time on a stage that is :merge-profile-only and opt-in, so the spend lands on a path already chosen. Re-run the corpus with STATIFIER_ADR_JUDGE_MODEL set to compare a candidate model before changing @default_model; see docs/testing.md's corpus subsection for the recorded scores and how to read a failure.

analyze/2 is pure given a source() (a diff already split into one in-scope slice per judged ADR) and an opts[:caller] (a function from a prompt string to a {:ok, response} | {:error, reason} tuple; real calls shell out to the claude CLI in production via call_claude_cli/1, a stub in tests). collect/1 gathers that source the same way Mix.Statifier.GateGuard and Mix.Statifier.AdrGuard do, plus two checks those guards do not need: whether the claude CLI is on PATH, checked before any git call runs (there is no point diffing if the stage cannot call out), and whether the diff touches any judged ADR's scope at all.

Local-only by design, and now purely so: call_claude_cli/1 shells out to the developer's own claude CLI (System.cmd/3) rather than calling the Anthropic API directly, so the stage rides the developer's existing Claude Code auth instead of needing its own ANTHROPIC_API_KEY. --tools "" and --strict-mcp-config keep the call a single non-agentic completion - no tool access, no MCP servers - so the judge can only read the prompt this module built, never the repository on its own.

Every parse failure or ambiguous model response fails closed rather than raising: an unparseable propose response yields no candidates, and an unparseable or ambiguous refute response is read as "not a violation" - the same tie-break the refute pass uses on a clear verdict. A candidate survives only when the refute pass explicitly says it does.

What is judged, and what is not

The @judged registry above covers three ADRs. Each is judge-shaped because its rule needs a model reading a change in context, not a name or call-site pattern a mechanical grep can match:

ADR-0012 (debuggability designed into the core) - whether a change drops a trace effect, a source location, or step counting/stamping is a question about what the change does, not what it is named.
ADR-0014 (expression-level spans) - extends ADR-0012 item 3 to expression granularity; whether a span table or an error's owning-node identity survives a refactor is the same kind of question, at finer grain.
ADR-0017 (judgment is not scriptable in wurk extensions), restating ADR-0015 constraint 4 for the surface that survives it - whether a wurk extension file rewrite quietly delegated a policy call to a script is exactly the propose/refute shape; ADR-0017 names this judge as the constraint's enforcement site.

Deliberately not covered:

ADR-0002 (literal Appendix D port) - the rubric is the Appendix D pseudocode itself, which is not in this repository; judging it would mean either pasting the spec into every prompt or letting the model judge from memory of it, the unverifiable verdict this design exists to avoid.
ADR-0003 (pure core with effects), ADR-0004 (predicator as the datamodel), ADR-0008 (generated identifier formats) - AdrGuard already covers each mechanically with a citation escape hatch; a model verdict over the same lines buys a second opinion on cases the guard already decides.
ADR-0005 (full configuration, interned state indexes) - the rule is a storage-shape choice visible in struct definitions, not in the hunks a --unified=0 diff shows; a judge reading hunks would be guessing at whole-module structure.
ADR-0011 (quality gate config is not agent-editable) - gate_guard's job mechanically; a probabilistic verdict does not belong in the path of the gate's own tamper check.
ADR-0015 constraints 1, 2, 3, and 5 - constraint 1's enforcement site (.claude/scripts/test/contract_test.rb) was removed along with the .claude/scripts/ tree it guarded once the kit's mechanics moved to another repo; 2, 3, and 5 were covered by the same script suite. ADR-0015 says plainly that re-enforcing constraint 1 elsewhere weakens it, so the same reasoning still rules out judging it here even though nothing enforces it in this repo today.
ADR-0001, ADR-0006, ADR-0007, ADR-0009, ADR-0010, ADR-0013 - process decisions (ADR format, the regression ratchet, beads, the gate itself, worktrees, repo archival) with no code shape a diff could violate. The prefix is repeated on each rather than elided after the first, so that searching this survey for "ADR-0006" finds the answer to "why is that one not judged?" - which is how the survey is meant to be read.

The reasoning above is the survey's conclusion for each candidate ADR. AdrGuard's moduledoc records the mirror-image reasoning for the mechanically-checkable ADRs - the two modules are meant to be read as a pair.

Summary

Types

caller()

candidate()

finding()

judged_source()

scope()

source()

Functions

analyze(source, opts \\ [])

Turns a source's in-scope diff slices, one per judged ADR, into adversarially-verified findings.

call_claude_cli(prompt)

The default opts[:caller]: one non-agentic claude CLI completion.

collect(opts)

Reads the diff and every judged ADR's text the judge needs.

in_scope?(path, scope)

Whether path falls inside scope: a prefix match, and an ends_with? match on scope.suffix too when it is non-nil (a nil suffix matches any path with the prefix, which is every scope registered today).

judged()

The judged-ADR registry, in the order findings are proposed.

parse_cli_response(output)

Turns claude --output-format json's stdout into call_claude_cli/1's return value.

refuse_real_call(prompt)

The :test-build default opts[:caller]: raises instead of shelling out.

scope_descriptions()

The human-readable scope of every judged ADR, in registry order.

scoped_chunks(diff, scope)

Splits a raw unified diff into one {path, chunk} pair per touched file, then keeps only the files in_scope?/2 accepts for scope. Each chunk is that file's whole diff text - context lines, removals, and additions alike, not only added lines - because a judge-shaped ADR violation (a dropped trace call, say) is as likely to be a removed line as an added one. Returns [] when the diff touches no file inside scope.

Types

caller()

@type caller() :: (String.t() -> {:ok, String.t()} | {:error, term()})

candidate()

@type candidate() :: %{
  file: String.t(),
  line: pos_integer() | nil,
  claim: String.t(),
  key: String.t(),
  label: String.t(),
  adr_text: String.t(),
  hunks: String.t()
}

finding()

@type finding() :: %{
  file: String.t(),
  line: pos_integer() | nil,
  severity: String.t(),
  check: String.t(),
  message: String.t()
}

judged_source()

@type judged_source() :: %{
  key: String.t(),
  label: String.t(),
  focus: String.t(),
  adr_text: String.t(),
  chunks: [{String.t(), String.t()}]
}

scope()

@type scope() :: %{prefix: String.t(), suffix: String.t() | nil, describe: String.t()}

source()

@type source() :: %{diff: String.t(), adrs: [judged_source()]}

Functions

analyze(source, opts \\ [])

@spec analyze(source :: source(), opts :: keyword()) :: [finding()]

Turns a source's in-scope diff slices, one per judged ADR, into adversarially-verified findings.

opts[:caller] defaults to call_claude_cli/1, a real shell-out to the claude CLI, in every build except :test - there it defaults to refuse_real_call/1, which raises. Tests always inject a stub caller anyway, since this is the one seam the whole module exists to keep pure; the :test default exists as a backstop for a test that forgets to.

call_claude_cli(prompt)

@spec call_claude_cli(prompt :: String.t()) :: {:ok, String.t()} | {:error, term()}

The default opts[:caller]: one non-agentic claude CLI completion.

--tools "" and --strict-mcp-config strip every tool and MCP server from the child session, so the prompt this module built is the only thing the call can see - no reading the repository, no reaching out on its own. --output-format json makes the reply parseable instead of scraping terminal prose, and --model carries STATIFIER_ADR_JUDGE_MODEL (falling back to @default_model) the same way the direct-API version did.

Only its shell-out half is unexercised by the test suite; every test injects its own caller, so this is the only place in the module that spawns a process. parse_cli_response/1, the half that turns the CLI's stdout into this function's return value, is pure and directly tested.

collect(opts)

@spec collect(opts :: keyword()) ::
  {:ok, source()}
  | {:error, String.t()}
  | :no_base_ref
  | :no_cli
  | :no_scoped_changes

Reads the diff and every judged ADR's text the judge needs.

Checked in order: opts[:cli_available] (falling back to System.find_executable/1) first, since there is nothing to gain from touching git when the stage has no way to call out; then base-ref resolution (opts[:base], then origin/main, then main), mirroring Mix.Statifier.AdrGuard; then whether the diff touches any registered scope at all. Each unmet condition returns its own atom so the task can report a distinct skip reason instead of a single opaque one.

opts[:runner] replaces the git shell-out with a function of an argument list returning {output, status}, mirroring Mix.Statifier.AdrGuard.

in_scope?(path, scope)

@spec in_scope?(path :: String.t(), scope :: scope()) :: boolean()

judged()

@spec judged() :: [map()]

The judged-ADR registry, in the order findings are proposed.

Public so tests can assert scoping (scoped_chunks/2) and prompt content against a real registry entry's scope/adr_path rather than a path string re-typed at the call site.

parse_cli_response(output)

@spec parse_cli_response(output :: String.t()) ::
  {:ok, String.t()} | {:error, String.t()}

Turns claude --output-format json's stdout into call_claude_cli/1's return value.

The CLI prints a JSON array of stream events; the one this cares about has "type": "result", and "is_error": false is the only shape read as success - anything else (a malformed array, no result event, an error result) fails closed to {:error, _} rather than guessing at partial text. Pure and directly testable with a fabricated array - no real CLI needed to exercise it.

refuse_real_call(prompt)

@spec refuse_real_call(prompt :: String.t()) :: no_return()

The :test-build default opts[:caller]: raises instead of shelling out.

Every test in this suite injects its own opts[:caller] stub; a test that forgets to is a bug, and the fix is to inject one, not to make a real claude CLI call and a real charge on that test's behalf. See the @default_caller moduledoc comment for the incident this guards against.

scope_descriptions()

@spec scope_descriptions() :: [String.t()]

The human-readable scope of every judged ADR, in registry order.

mix adr.judge's skip reason joins these so the reason for "nothing to judge" names every scope the registry actually checks, with one definition site instead of the string being written again in the task. Deduped, so two registry entries sharing one scope (as ADR-0012 and ADR-0014 do today) name it once rather than repeating it.

scoped_chunks(diff, scope)

@spec scoped_chunks(diff :: String.t(), scope :: scope()) :: [
  {String.t(), String.t()}
]