mix aludel.eval (aludel v0.7.0)

Copy Markdown View Source

Runs one Aludel evaluation suite and emits a report.

mix aludel.eval --file path/to/suite.yaml

A schema-version-1 JSON or YAML manifest can identify the persisted suite, prompt version, provider, and optional repeated-sampling settings. Individual target flags remain available:

mix aludel.eval       --suite-id SUITE_ID       --prompt-version-id PROMPT_VERSION_ID       --provider-id PROVIDER_ID       --format json

Supported formats are json (the default), console, junit, and github. Use --output PATH to write the report to a file instead of standard output. JSON output uses the versioned Aludel report schema. JUnit omits generated responses unless --include-output is set.

The task exits unsuccessfully when arguments or targets are invalid, execution cannot complete, or the active suite quality policy does not pass. Suites without a quality policy retain the all-test-cases-must-pass behavior.