Spectre.Governance.EvaluationDelta (Spectre v0.3.0)

Deterministic comparison of parent and Candidate results on the same corpus.

Parent/Candidate score deltas are computed only from protected cases. Cases introduced by the current Candidate are tracked separately and must pass, but can never improve its score or compensate for a protected regression. Protected identity covers each complete canonical Spectre.Eval.Case, not only its id, so changing an input, oracle, policy, state, or tag changes the corpus digest.

Summary

Types

result()

t()

Functions

digest(delta)

Returns the canonical evaluation-delta digest.

from_data(data)

Restores and recomputes an evaluation delta from portable data.

new(parent_results, candidate_results, opts \\ [])

Builds an evaluation delta from parent and Candidate result lists.

new!(parent_results, candidate_results, opts \\ [])

Builds an evaluation delta or raises with the stable reason.

protected_corpus_digest(cases)

Computes the canonical digest of complete protected case content.

protected_corpus_digest!(cases)

Computes the protected-corpus digest or raises with the stable reason.

to_data(delta)

Returns JSON-shaped evaluation evidence.

Types

result()

@type result() :: %{required(String.t()) => String.t() | boolean() | number()}

t()

@type t() :: %Spectre.Governance.EvaluationDelta{
  candidate_case_failures: [String.t()],
  candidate_cases_digest: String.t(),
  candidate_owned_results: [map()],
  candidate_score: float(),
  max_regressions: non_neg_integer(),
  min_score_delta: float(),
  parent_score: float(),
  passed: boolean(),
  protected_cases: [map()],
  protected_cases_digest: String.t(),
  protected_results: [map()],
  regressions: [String.t()],
  schema_version: pos_integer(),
  score_delta: float()
}

Functions

digest(delta)

@spec digest(t()) :: String.t()

Returns the canonical evaluation-delta digest.

from_data(data)

@spec from_data(map()) :: {:ok, t()} | {:error, term()}

Restores and recomputes an evaluation delta from portable data.

new(parent_results, candidate_results, opts \\ [])

@spec new([map()], [map()], keyword()) :: {:ok, t()} | {:error, term()}

Builds an evaluation delta from parent and Candidate result lists.

new!(parent_results, candidate_results, opts \\ [])

@spec new!([map()], [map()], keyword()) :: t()

Builds an evaluation delta or raises with the stable reason.

protected_corpus_digest(cases)

@spec protected_corpus_digest([Spectre.Eval.Case.t() | map()]) ::
  {:ok, String.t()} | {:error, term()}

Computes the canonical digest of complete protected case content.

protected_corpus_digest!(cases)

@spec protected_corpus_digest!([Spectre.Eval.Case.t() | map()]) :: String.t()

Computes the protected-corpus digest or raises with the stable reason.

to_data(delta)

@spec to_data(t()) :: map()

Returns JSON-shaped evaluation evidence.