adk_eval (erlang_adk v0.2.1)

adk_eval - Evaluation framework for ADK 2.0.

Allows defining datasets and running them against an agent to calculate metrics.

Summary

dataset/0

metric_fn/0

Run an evaluation.

Run an evaluation with options (e.g. concurrency).

-type dataset() :: [#{input => term(), expected => term(), metadata => map()}].

-type metric_fn() :: fun((Expected :: term(), Actual :: term()) -> float()).

-spec run(AgentRef :: term(), Dataset :: dataset(), MetricFn :: metric_fn()) ->
             {ok, map()} | {error, term()}.

Run an evaluation.

-spec run(AgentRef :: term(), Dataset :: dataset(), MetricFn :: metric_fn(), Opts :: map()) ->
             {ok, map()} | {error, term()}.

Run an evaluation with options (e.g. concurrency).