@spec calibration_report(
  [map()],
  keyword()
) :: map()

Join bank-scoped human and LLM calibration rows and evaluate the quadratic-kappa gate.

The confusion matrix is a 4x4 nested list whose rows are human grades and columns are LLM grades. Undefined kappa always fails the gate.

Options:

• :bank_id - restrict the report to one bank
• :threshold - pass threshold (default: 0.6)

@spec gold_judge_pairs(map()) :: [map()]

Convert every gold-artifact row to the atom-keyed pair shape accepted by Judge.

Calibration and test splits are preserved. Existing human grades are deliberately not copied into judge input.

@spec judge_pairs(map(), map(), [atom()]) :: [map()]

Build the complete judge set and retain only the requested splits.

This supports correction passes that must replace calibration and train labels without touching held-out test labels.

@spec prefill_test_artifact(
  map(),
  [map()] | %{required(atom() | String.t()) => term()}
) :: map()

Prefill successful LLM grades onto test rows without revealing calibration judgements.

The artifact is otherwise preserved. Test rows with no matching successful label remain unchanged. Both a label list and the %{labels: labels} result returned by Judge are accepted.

@spec train_judge_pairs(map(), map()) :: [map()]

Build the bulk train judge set from a query-pool sidecar.

Every pair present in the gold artifact is excluded, and duplicate candidates are collapsed by {query_ref, memory_ref}. All resulting pairs have the :train split.