Deterministic assertions that don't require LLM calls.
Fast, free, and should run first before expensive LLM-based checks.
Summary
Functions
Evaluates a deterministic assertion.
Functions
Evaluates a deterministic assertion.
Assertion Types
:contains- Output contains substring(s):not_contains- Output does not contain substring(s):contains_any- Output contains at least one of the substrings:contains_all- Output contains all substrings:regex- Output matches regex pattern:is_json- Output is valid JSON:max_tokens- Output is under token limit (approximated by words):latency_ms- Response was within time limit
Examples
evaluate(:contains, "Hello world", value: "world")
#=> {:pass, %{matched: "world"}}
evaluate(:regex, "Price: $29.99", value: ~r/\$\d+\.\d{2}/)
#=> {:pass, %{matched: "$29.99"}}
evaluate(:is_json, ~s({"name": "test"}), [])
#=> {:pass, %{parsed: %{"name" => "test"}}}