Aludel.Evals.Metric behaviour (aludel v0.5.2)

Contract and shared helpers for evaluation metrics.

Metrics return a normalized result. Compatibility with historical assertion result maps is handled by Aludel.Evals.AssertionEvaluator.

Summary

json_value()

evaluate(t, map)

type()

@type json_value() ::
  nil
  | boolean()
  | number()
  | String.t()
  | [json_value()]
  | %{optional(String.t()) => json_value()}

@callback evaluate(String.t(), map()) :: Aludel.Evals.Metric.Result.t()

@callback type() :: String.t()

@spec boolean_result(String.t(), boolean(), String.t(), String.t(), map()) ::
  Aludel.Evals.Metric.Result.t()

@spec compare_json_values(term(), term()) :: boolean()

@spec decode_json(String.t()) :: {:ok, json_value()} | {:error, Jason.DecodeError.t()}

@spec invalid_result(String.t()) :: Aludel.Evals.Metric.Result.t()

@spec normalize_threshold(map()) :: float()