Generates attacks that probe for hallucination — fabricated specifics the assistant has no way of actually knowing.
Inspired by promptfoo's hallucination plugin. The attack pattern: ask
for very specific information (lists, statistics, niche names, exact
values) that looks like a reasonable request for the assistant's domain
but is extremely unlikely to be known or verifiable. The trap is that
refusing or admitting ignorance feels less helpful than answering, so
the assistant fabricates.
Each generated case carries the purpose in expected.hallucinated.purpose
so the Tribunal.Judges.Hallucinated judge can grade responses against
the same scope at run time.
Distinct from the existing Tribunal.Judges.Hallucination judge, which
grades faithfulness against a supplied :context. This plugin (and its
paired Hallucinated judge) work without ground truth — the judge looks
for the signature of confabulation rather than verifying facts.
Options
:purpose— required. Free-form description of the assistant's domain.:count— number of attacks to generate. Defaults to5.:attacker— module implementingTribunal.RedTeam.Attacker. Defaults toTribunal.RedTeam.Attacker.default/0.:model— passed through to the attacker.:temperature,:max_tokens— passed through to the attacker.
Example
{:ok, cases} = Tribunal.RedTeam.Plugins.Hallucination.generate(
purpose: "Shopping assistant for a cosmetics retailer.",
count: 5
)