mix just_bash.eval (JustBash v0.4.0)

View Source

Runs the JustBash agent eval suite.

Usage

mix just_bash.eval                       # Run all evals
mix just_bash.eval --task jq_transform   # Run a specific eval
mix just_bash.eval --retries 3           # Retry failing tasks up to 3 times
mix just_bash.eval --concurrency 8       # Run 8 tasks concurrently
mix just_bash.eval --no-persist          # Skip writing results to JSONL
mix just_bash.eval --verbose             # Verbose output (default: true)

Results are written to eval_results/results.jsonl by default. Requires ANTHROPIC_API_KEY in environment or config.