Assertion evaluation engine.
Routes assertions to the appropriate implementation:
- Deterministic:
contains,regex,is_json, etc. - Judge (requires req_llm):
faithful,relevant, etc. - Embedding (requires alike):
similar
Summary
Functions
Checks if all assertions passed.
Returns list of available assertion types based on loaded dependencies.
Evaluates a single assertion against a test case.
Evaluates multiple assertions against a test case.
Evaluates assertions in order without collapsing repeated assertion types.
Functions
Checks if all assertions passed.
Returns list of available assertion types based on loaded dependencies.
Evaluates a single assertion against a test case.
Returns {:pass, details} or {:fail, details}.
Evaluates multiple assertions against a test case.
Returns a map of %{assertion_type => result}.
Evaluates assertions in order without collapsing repeated assertion types.
Default options are applied to every assertion. Options declared on the assertion take precedence over defaults.