Core of the mix noizu.mcp.eval description-tuning harness (spec §4).

For every selected (tool, eval, verbosity permutation), the harness renders the tool's wire schema through the §0/§2/§3 resolution pipeline (Noizu.MCP.Types.Tool.to_map/2) with the permutation's RenderCtx — this is the point of the whole exercise: evals exercise the rendered descriptions. It then runs the eval prompt against a target via a Noizu.MCP.Eval.Runner and grades each rubric criterion via a Noizu.MCP.Eval.Judge, collecting a JSON -ready report.

run/1 options

• :server (required) — the Noizu.MCP.Server module to eval
• :tool — restrict to one tool by wire name (default: all tools with evals)
• :verbosity — 0..9, :all (permute over 0..9), or nil (a single permutation at the server/global default level)
• :runner — a RenderCtx runner atom (e.g. :codex) applied to every permutation; not to be confused with the runner adapter below
• :model — a RenderCtx model (atom or string) applied to every permutation
• :runner_adapter — the Noizu.MCP.Eval.Runner module (default: :noizu_mcp :eval_runner app env, else Noizu.MCP.Eval.Runner.Stub)
• :judge_adapter — the Noizu.MCP.Eval.Judge module (default: :noizu_mcp :eval_judge app env, else Noizu.MCP.Eval.Judge.Stub)

Report shape

%{
  "server" => "...", "runner_adapter" => "...", "judge_adapter" => "...",
  "runner" => "codex" | nil, "model" => "5.4" | nil,
  "generated_at" => iso8601,
  "summary" => %{"total" => n, "passed" => n, "failed" => n},
  "results" => [
    %{
      "tool" => "...", "eval" => "...", "verbosity" => 0..9 | nil,
      "runner" => ..., "model" => ..., "description" => rendered_text,
      "runner_status" => "ok" | "error",
      "criteria" => [%{"criterion" => "...", "pass" => bool,
                       "score" => float, "notes" => "..."}],
      "pass" => bool
    }
  ]
}

A result fails when its runner errors or any of its rubric criteria fail; summary.failed counts failing results — the signal mix noizu.mcp.eval --gate turns into a non-zero exit.