Spectre.Eval.Case (Spectre v0.3.0)

Copy Markdown View Source

One version-controlled routing expectation.

Cases can be created from ordinary maps or decoded JSONL objects. Labels and strategies remain strings so loading an external corpus never creates atoms.

Summary

Types

t()

Functions

expected_routes(evaluation_case)

Returns all accepted route labels for this case in canonical form.

new(evaluation_case)

Validates and normalizes a case map.

to_data(evaluation_case)

Returns the complete portable shape used to identify an evaluation case.

Types

t()

@type t() :: %Spectre.Eval.Case{
  allowed_routes: [String.t()],
  context: map(),
  expected_outcome: Spectre.Router.Receipt.outcome(),
  expected_output: String.t() | nil,
  expected_route: String.t() | nil,
  expected_strategy: String.t() | nil,
  id: String.t(),
  input: String.t() | map(),
  llm: :forbidden | :allowed | :required,
  max_duration_us: pos_integer() | nil,
  state: map() | keyword(),
  tags: [String.t()]
}

Functions

expected_routes(evaluation_case)

@spec expected_routes(t()) :: [String.t()]

Returns all accepted route labels for this case in canonical form.

new(evaluation_case)

@spec new(t() | map()) :: {:ok, t()} | {:error, term()}

Validates and normalizes a case map.

to_data(evaluation_case)

@spec to_data(t()) :: map()

Returns the complete portable shape used to identify an evaluation case.