adk_eval_statistics (erlang_adk v0.10.0)

View Source

Deterministic confidence and longitudinal regression helpers.

The module deliberately implements a small, auditable statistical surface: Wilson intervals for pass rates and a normal-approximation interval for the difference between two bounded score series. It never samples randomness, so the same stored evaluation results produce byte-stable gate decisions.

Summary

Functions

confidence_interval(Samples, Options)
longitudinal_gate(Baseline0, Current0, Options)

Compare two bounded score series.

pass_rate_interval(Passed, Total, Options)
summary(Samples)

Functions

confidence_interval(Samples, Options)

-spec confidence_interval([number()], map()) -> {ok, map()} | {error, term()}.

longitudinal_gate(Baseline0, Current0, Options)

-spec longitudinal_gate([number()], [number()], map()) -> {ok, map()} | {error, term()}.

Compare two bounded score series.

Options: * confidence_level - 0.90, 0.95 (default), or 0.99 * max_mean_drop - tolerated baseline minus current mean, 0..1 * require_significance - when true (default), a regression is reported only when the one-sided confidence bound also exceeds the tolerance.

pass_rate_interval(Passed, Total, Options)

-spec pass_rate_interval(non_neg_integer(), non_neg_integer(), map()) -> {ok, map()} | {error, term()}.

summary(Samples)

-spec summary([number()]) -> {ok, map()} | {error, term()}.