ExAgent.Patterns.Reflection (ExAgent v0.4.0)

Copy Markdown View Source

Draft, critique, revise - until a reviewer signs off or the budget runs out.

The analogy: a writer and an editor. The writer produces a draft. The editor marks it up. The writer revises. Two or three rounds of that beats one person trying to be brilliant on the first attempt.

Use it when quality is the bottleneck and you can say what good looks like: code that has to compile, a summary that must not exceed 200 words, SQL that must only read from three tables. Reflection is the pattern that turns "usually fine" into "checked".

alias ExAgent.Patterns.Reflection

{:ok, result} =
  Reflection.run("Write a Postgres query for monthly active users.",
    generator: coder,
    critic: reviewer,
    accept?: &String.contains?(&1, "APPROVED"),
    max_rounds: 3
  )

result.output   # the accepted draft
result.rounds   # how many revisions it took

The critic needs a way to say yes

accept? decides when to stop. Ask the critic for a token you can match on and keep the instruction blunt - "Reply APPROVED if it is correct, otherwise list what is wrong". Without a stop condition the loop just burns max_rounds worth of tokens every time.

Two guardrails, both deliberate

max_rounds defaults to 3 and is a hard ceiling: an LLM critic can always find something to complain about, so an unbounded loop is a runaway bill.

When the ceiling is hit without approval you get {:max_rounds, result}, not {:ok, result}. The last draft is still there - often it is good enough - but you have to choose to use unapproved work rather than have it handed to you as if a reviewer had passed it.

When not to use it

If you cannot describe the acceptance criteria, a critic will produce vague praise and you will pay double for the same answer. If the risk is a wrong fact rather than poor quality, prefer ExAgent.Patterns.Consensus - a critic reading one draft is easy to talk into agreeing with it.

Summary

Types

reflection_opts()
result()
target()

Where to send a prompt: a provider struct for a stateless call, or a running agent when the step should remember the conversation.

Functions

run(task, opts)

Generates an answer to task, then critiques and revises it.

Types

reflection_opts()

@type reflection_opts() :: [
  generator: target(),
  critic: target(),
  accept?: (String.t() -> boolean()),
  max_rounds: pos_integer(),
  critic_prompt: (String.t(), String.t() -> String.t()),
  revise_prompt: (String.t(), String.t(), String.t() -> String.t())
]

result()

@type result() :: %{
  output: String.t(),
  rounds: non_neg_integer(),
  critiques: [String.t()],
  accepted?: boolean()
}

target()

@type target() :: struct() | GenServer.server()

Where to send a prompt: a provider struct for a stateless call, or a running agent when the step should remember the conversation.

Functions

run(task, opts)

@spec run(String.t(), reflection_opts()) ::
  {:ok, result()} | {:max_rounds, result()} | {:error, term()}

Generates an answer to task, then critiques and revises it.

Options

  • :generator (required) - provider or agent that drafts and revises
  • :critic - provider or agent that reviews (default: the generator, which is cheaper but a weaker check - a model reviewing itself agrees with itself)
  • :accept? - (critique -> boolean) deciding when the draft is good (default: the critique contains "APPROVED")
  • :max_rounds - hard ceiling on revisions (default: 3)
  • :critic_prompt - (task, draft -> prompt) to override the review prompt
  • :revise_prompt - (task, draft, critique -> prompt) to override the revision

Returns {:ok, result} once the critic accepts, {:max_rounds, result} if it never does, or {:error, reason} if a call fails.