Aludel.Evals.QualityPolicy (aludel v0.7.0)

Copy Markdown View Source

Validates and evaluates versioned suite quality policies.

Policies combine pass-rate, metadata-group, evaluator-score, cost, and latency rules. Evaluation returns explicit passed, failed, invalid, or unavailable status without conflating missing evidence with a quality failure.

Summary

Types

definition()
evaluation()

Functions

evaluate(definition, results, summary)

Evaluates a policy against persisted test case results and suite aggregates.

validate(definition)

Validates a version-one policy definition.

Types

definition()

@type definition() :: %{required(String.t()) => term()}

evaluation()

@type evaluation() :: %{required(String.t()) => term()}

Functions

evaluate(definition, results, summary)

@spec evaluate(term(), [map()], map()) :: evaluation()

Evaluates a policy against persisted test case results and suite aggregates.

Missing test cases, matching metadata groups, evaluator scores, cost, or latency produce an unavailable rule instead of silently passing or failing. Malformed definitions produce an invalid policy result.

validate(definition)

@spec validate(term()) :: :ok | {:error, [String.t()]}

Validates a version-one policy definition.

A policy must be JSON-compatible, cannot exceed 100000 encoded bytes, and must contain between one and 50 field-strict rules with unique IDs. Pass rates use values from 0.0 through 1.0, evaluator scores use values from 0 through 100, and budget maximums must be non-negative.