Aludel.Evals.Metric.Evaluator (aludel v0.7.0)

Normalized execution details for the evaluator behind a metric result.

Deterministic metrics usually record only status and duration. Model-backed evaluators can also attribute provider, model, token usage, and cost without mixing that usage with the model output being evaluated.

Summary

Types

status()

t()

Functions

completed(duration_ms, attrs \\ [])

Records a successfully completed evaluator.

error(error, attrs \\ [])

Records an evaluator infrastructure failure.

new(status, attrs \\ [])

Creates evaluator details and validates the lifecycle status.

to_map(evaluator)

Converts evaluator details to a JSON-compatible map, omitting absent values.

unavailable(error)

Records an evaluator that could not be resolved or used.

with_duration(evaluator, duration_ms)

Adds measured duration when the evaluator did not provide its own timing.

Types

status()

@type status() :: :completed | :error | :unavailable

t()

@type t() :: %Aludel.Evals.Metric.Evaluator{
  cost_usd: number() | nil,
  duration_ms: number() | nil,
  error: map() | nil,
  input_tokens: non_neg_integer() | nil,
  model: String.t() | nil,
  output_tokens: non_neg_integer() | nil,
  provider: String.t() | nil,
  status: status()
}

Functions

completed(duration_ms, attrs \\ [])

@spec completed(
  number(),
  keyword()
) :: t()

Records a successfully completed evaluator.

error(error, attrs \\ [])

@spec error(
  map(),
  keyword()
) :: t()

Records an evaluator infrastructure failure.

new(status, attrs \\ [])

@spec new(
  status(),
  keyword()
) :: t()

Creates evaluator details and validates the lifecycle status.

to_map(evaluator)

@spec to_map(t()) :: map()

Converts evaluator details to a JSON-compatible map, omitting absent values.

unavailable(error)

@spec unavailable(map()) :: t()

Records an evaluator that could not be resolved or used.

with_duration(evaluator, duration_ms)

@spec with_duration(t(), number()) :: t()

Adds measured duration when the evaluator did not provide its own timing.