Aludel.Evals.Metric.Context (aludel v0.7.0)

Evidence available to an evaluation metric.

The context keeps metric implementations independent from Ecto schemas while providing enough information for deterministic checks, model-based judges, and application-defined evaluators. Optional fields use JSON-compatible values so results can be reproduced and reported consistently.

Existing metrics that only need generated text may continue to accept a string. Use new/2 when a metric needs prompt, provider, document, or execution evidence.

Summary

Types

json_value()

t()

Functions

new(output, attrs \\ [])

Creates a metric context for generated output and optional evidence.

Types

json_value()

@type json_value() ::
  nil
  | boolean()
  | number()
  | String.t()
  | [json_value()]
  | %{optional(String.t()) => json_value()}

t()

@type t() :: %Aludel.Evals.Metric.Context{
  documents: [map()],
  execution: map(),
  expected: json_value(),
  messages: [map()],
  metadata: map(),
  output: String.t(),
  prompt_template: String.t() | nil,
  prompt_version: map() | nil,
  provider: map() | nil,
  rendered_input: String.t() | nil,
  variables: map()
}

Functions

new(output, attrs \\ [])

@spec new(
  String.t(),
  keyword()
) :: t()

Creates a metric context for generated output and optional evidence.