Ask the same question several times, then go with the answer that keeps coming up.
The analogy: a second opinion. You do not ask one doctor to explain their reasoning more carefully; you ask three doctors and see whether they agree.
Use it when being wrong is expensive and the answer is short enough to compare: a classification, a routing decision, an extracted number, a yes/no. Independent attempts catch the case where a model is confidently mistaken, which no amount of self-review will - a critic reading one draft is easily talked into agreeing with it.
alias ExAgent.Patterns.Consensus
{:ok, verdict} =
Consensus.run("Is this email a phishing attempt? Answer YES or NO.\n\n" <> body,
voters: provider,
samples: 5
)
verdict.answer # "YES"
verdict.agreement # 0.8 - four of five agreed
verdict.votes # %{"YES" => 4, "NO" => 1}Disagreement is the useful part
The point is not only the winning answer but how lopsided the vote was.
:agreement is the winner's share, and a low number is your signal to escalate
to a bigger model, to a human, or to declining to answer:
case Consensus.run(question, voters: provider, samples: 5) do
{:ok, %{agreement: a} = verdict} when a >= 0.8 -> {:auto, verdict.answer} {:ok, verdict} -> {:needs_review, verdict}end
Different voters beat repeated voters
Passing one provider samples it repeatedly, which only helps if its output
varies - at temperature: 0 you will pay five times for one answer. Passing a
list of providers is stronger, because different models fail differently:
Consensus.run(question, voters: [gpt, gemini, local_llama])Normalize before counting
Votes are tallied on exact strings after trimming and downcasing, so "YES" and
"Yes." are different answers unless you say otherwise. For anything but a
strict format, pass :normalize:
Consensus.run(question,
voters: provider,
samples: 5,
normalize: fn answer -> answer |> String.upcase() |> String.contains?("YES") end
)When not to use it
Long prose does not vote - two summaries are never byte-identical, so every
answer wins with 1/n. Use ExAgent.Patterns.Reflection to improve a long
answer, and consensus to decide a short one.
Summary
Types
Where to send a prompt: a provider struct for a stateless call, or a running agent when the step should remember the conversation.
Functions
Puts prompt to several voters and returns the most common answer.
Types
@type consensus_opts() :: [ voters: target() | [target()], samples: pos_integer(), normalize: (String.t() -> term()), timeout: pos_integer() ]
@type target() :: struct() | GenServer.server()
Where to send a prompt: a provider struct for a stateless call, or a running agent when the step should remember the conversation.
Functions
@spec run(String.t(), consensus_opts()) :: {:ok, verdict()} | {:error, {:no_answers, [term()]} | :no_voters}
Puts prompt to several voters and returns the most common answer.
Options
:voters(required) - one provider/agent, sampled:samplestimes, or a list of them, each asked once:samples- how many times to ask a single voter (default:3); ignored when:votersis a list:normalize-(answer -> comparable)applied before votes are counted (default: trim and downcase):timeout- per voter, in milliseconds (default: 5 minutes)
Returns {:ok, verdict}, {:error, {:no_answers, failures}} if every voter was
asked and every one failed, or {:error, :no_voters} if there was nobody to ask
(an empty :voters list, or a non-positive :samples). Ties
are broken by first appearance, so the winner is always the earliest of the
joint-highest - deterministic, but check :agreement before trusting a tie.