mix arcana.eval.run (Arcana v2.0.1)

Copy Markdown View Source

Runs evaluation and prints metrics.

Usage

mix arcana.eval.run
mix arcana.eval.run --mode hybrid
mix arcana.eval.run --generate --sample-size 50
mix arcana.eval.run --format json > results.json

Options

  • --mode - Search mode: semantic, fulltext, hybrid (default: semantic)
  • --source-id - Limit to specific source
  • --generate - Generate test cases first if none exist
  • --sample-size - Sample size for generation (default: 50)
  • --format - Output format: table, json (default: table)
  • --fail-under - Exit 1 if recall@5 below threshold (for CI)

Configuration

Requires :repo to be configured in your application:

config :arcana, repo: MyApp.Repo

If using --generate, also requires :llm to be configured.