mix tribunal.redteam.generate (Tribunal v1.4.0)

Copy Markdown View Source

Generates a red-team dataset by invoking one or more plugins.

Generation is separate from running. The output is a regular Tribunal dataset file that you execute with mix tribunal.eval or tribunal_eval/2, the same as any hand-written eval suite.

Usage

mix tribunal.redteam.generate \
  --plugins policy \
  --purpose-file priv/purpose.txt \
  --policy-file priv/policy.txt \
  --count 5 \
  --output test/evals/datasets/redteam.yaml

Options

  • --plugins - Required. Comma-separated plugin ids (e.g., policy).
  • --purpose - Inline purpose text. Mutually exclusive with --purpose-file.
  • --purpose-file - Path to a file containing the purpose text.
  • --policy - Inline policy text.
  • --policy-file - Path to a file containing the policy text.
  • --count - Cases per plugin (default: 5).
  • --output - Path to write the dataset. Format inferred from extension (.yaml/.yml → YAML, .json → JSON). Defaults to stdout in YAML.
  • --format - Override format detection (yaml or json).
  • --model - Attacker LLM model spec, passed through to the attacker.

Example

mix tribunal.redteam.generate \
  --plugins policy \
  --purpose "Shopping assistant for a cosmetics retailer." \
  --policy-file priv/policy.txt \
  --count 5 \
  --output test/evals/datasets/policy_redteam.yaml