Arcana.Loop (Arcana v4.0.0)

Agentic RAG via an LLM-driven tool loop.

Where Arcana.Pipeline composes RAG steps that you decide ahead of time, Arcana.Loop lets the LLM decide what to do each turn. The controller picks tools (search, answer, give_up) until it has enough context to answer the question or hits a safety limit.

This is the "Agentic RAG" pattern from Singh et al.'s 2025 survey: an LLM driven loop with tool use, as opposed to the static "Modular RAG" pipeline in Arcana.Pipeline. Both patterns are useful and they coexist.

Quick start

{:ok, ctx} =
  Arcana.Loop.new("Find episodes where a Time Lord betrayed the Doctor",
    repo: MyApp.Repo,
    collection: "doctor-who"
  )
  |> Arcana.Loop.run(controller_llm: "openai:gpt-4o-mini")

ctx.answer
ctx.tool_history
ctx.terminated_by

The loop's LLM options differ from the rest of arcana

Arcana.ask/2, Arcana.Pipeline, graph extraction and community summarization all accept a {module, function} LLM. The loop does not, and passing one raises ArgumentError. Two things to know before adopting it.

req_llm is required either way. run/2 refuses to start without it, because the messages and tools the loop builds are ReqLLM.Context and ReqLLM.Tool structs. So the loop is the one part of arcana an app cannot use while leaving req_llm out.

You can still route the model call through your own stack, using a three-arity function rather than a {module, function} tuple:

Arcana.Loop.run(ctx,
  controller_llm: fn messages, tools, opts ->
    # messages are ReqLLM.Context entries, tools are ReqLLM.Tool structs.
    # Return the shape ReqLLM.Response.classify/1 produces. The loop
    # routes on :type and has no fallback clause, so it must be there:
    #
    #     {:ok, %{type: :tool_calls, tool_calls: [...]}}
    #     {:ok, %{type: :final_answer, text: "..."}}
    #     {:error, reason}
    MyApp.LLM.complete_with_tools(messages, tools, opts)
  end
)

That is the hook for your own tracing and cost accounting. What it cannot do is live in config/runtime.exs as data: a captured function does not serialize into a release's sys.config, which is exactly what {module, function} was added for elsewhere. So the loop's LLM has to be wired where code runs, not in config.

Why the tuple is refused rather than supported: a {module, function} LLM is called as a plain completion - prompt in, text out - with nowhere to pass tools or read tool calls back. The loop needs both, so accepting the shape would mean a provider-agnostic tool-calling contract for arcana. If you want that, say so on #164.

:answer_llm carries the same restriction. The answerer is passed no tools and asked for prose, but both roles go through the same call path, so it takes a model spec or a three-arity function too.

Its return contract is looser, though. The :type routing above is the controller's: the answerer only needs {:ok, %{text: binary}} with a non-empty string, and anything else - including an error - silently keeps the controller's own draft rather than failing the run.

The max_iterations fallback synthesizer has no draft underneath it at all, since it only runs when the controller never called answer, so failing it costs you more. A failed synthesis leaves :answer as nil and the run still comes back {:ok, ctx} with terminated_by: :max_iterations.

The non-empty part of that contract belongs to the default synthesizer rather than to the loop. A custom :synthesizer has its {:ok, text} stored verbatim, so {:ok, ""} leaves you an empty answer instead of nil. If you supply your own, check that the answer is present rather than just non-nil.

Two models

You can configure separate models for the loop controller and the final answer. When :answer_llm is omitted the controller is used for both.

Termination

The loop terminates on any of:

The controller calls the answer tool. terminated_by: :answered.
The controller calls the give_up tool. terminated_by: :gave_up.
max_iterations is reached. terminated_by: :max_iterations.
The controller LLM returns an error. terminated_by: :error.
The controller returns a final answer with no tool calls. terminated_by: :answered.

Configuration

Set defaults via app config. Per-call options override these.

config :arcana, loop: [
  max_iterations: 10,
  controller_llm: "openai:gpt-4o-mini",
  chunk_cap: 30
]

Telemetry

Each phase emits events under [:arcana, :loop, :*].

References

Summary

Functions

ground(ctx, opts \\ [])

Runs a grounding analysis on the loop's answer against the accumulated chunks and stores the result in ctx.grounding.

new(question, opts \\ [])

Builds a new Arcana.Loop.Context for run/2.

run(ctx, opts \\ [])

Runs the agent loop until it terminates.

Functions

ground(ctx, opts \\ [])

@spec ground(
  Arcana.Loop.Context.t(),
  keyword()
) :: Arcana.Loop.Context.t()

Runs a grounding analysis on the loop's answer against the accumulated chunks and stores the result in ctx.grounding.

Reuses the Arcana.Grounder behaviour. By default uses Arcana.Grounder.Hallmark, which scores sentence-level faithfulness via Vectara HHEM through Bumblebee.

Skipped cases

Grounding is a no-op (ctx.grounding stays nil) when:

The loop terminated with terminated_by: :error (ctx.error is set)
ctx.answer is nil
ctx.chunks is empty — there's nothing to ground against

Options

:grounder - Module implementing Arcana.Grounder or a 3-arity function (answer, chunks, opts) -> {:ok, result} | {:error, reason}. Defaults to Arcana.Grounder.Hallmark. Use Arcana.Grounder.LLMJudge for LLM-as-judge faithfulness instead of NLI scoring.

Any other options are passed through to the grounder, with :question added automatically from ctx.question.

Example

{:ok, ctx} =
  Arcana.Loop.new("What is a TARDIS?", repo: repo, collection: "doctor-who")
  |> Arcana.Loop.run(controller_llm: llm)

ctx = Arcana.Loop.ground(ctx)

ctx.grounding.score               # 0.0-1.0 faithfulness
ctx.grounding.hallucinated_spans  # unsupported sentences
ctx.grounding.faithful_spans      # supported sentences + chunk attribution

Errors from the grounder are swallowed: if the grounder fails, ctx.grounding stays nil and the rest of the context is unchanged. Grounding is a nice-to-have annotation, not a fatal step.

new(question, opts \\ [])

@spec new(
  String.t(),
  keyword()
) :: Arcana.Loop.Context.t()

Builds a new Arcana.Loop.Context for run/2.

Options

:repo - Ecto repo for retrieval tools. Falls back to :repo global config.
:collection - Collection scope as :all, a name, or a list of names.

run(ctx, opts \\ [])

@spec run(
  Arcana.Loop.Context.t(),
  keyword()
) :: {:ok, Arcana.Loop.Context.t()}

Runs the agent loop until it terminates.

Returns {:ok, ctx} with the final loop context. The context's :terminated_by, :answer, :tool_history, and :chunks fields tell you what happened.

Options

:tools - List of ReqLLM.Tool structs. Defaults to Tools.default/0.
:max_iterations - Hard cap on controller turns. Default 10.
:controller_llm - Model spec for the loop controller. Required. Either a ReqLLM model string or a function fn messages, tools, opts -> {:ok, classified} | {:error, reason} end.
:answer_llm - Optional model spec for the answerer, separate from the controller. When set, the loop's controller picks tools as usual, but the user-facing answer text is produced by the answer_llm via a separate tool-less LLM call. Use this for the "cheap controller / strong answerer" pattern: a small fast model drives the loop, a stronger model writes the final answer. Triggers on the answer tool path (rewrites the controller's draft) and on the max_iterations synthesis fallback (used as the default synthesizer). Does not rewrite give_up (which would just dress up failure in nicer prose). When unset, the controller's text is used as ctx.answer directly.
:system_prompt - Override the default system prompt. Either a string or a function fn opts -> string end.
:chunk_cap - Maximum chunks accumulated across iterations. Default 30.
:search_fn - Override Arcana.search/2 for the built-in search tool. Used in tests; receives (query, search_opts).
:search_opts - Extra options forwarded to the search tool's call into Arcana.search/2.
:fallback_synthesis - When the loop hits max_iterations without answer being called and chunks have been accumulated, do one final tool-less LLM call to synthesize an answer from those chunks. Defaults to true. Set to false to leave ctx.answer as nil on max_iterations.
:synthesizer - Override the synthesis function used for the max_iterations fallback path. Receives (messages, opts) and must return {:ok, text} or {:error, reason}. When unset, the default synthesizer calls :answer_llm if set, otherwise :controller_llm. Setting :synthesizer directly bypasses both. Note: this only affects the fallback path. The answer tool rewrite always uses :answer_llm directly when set, regardless of :synthesizer.
:answer_prompt - Override the instruction text appended to the conversation when :answer_llm rewrites the controller's draft answer. String (literal text) or (opts -> string) function. Use this to control answer style without replacing the whole :answer_llm. Example: answer_prompt: "Write a one-paragraph summary. No bullet points."
:synthesis_prompt - Same as :answer_prompt but for the max_iterations fallback synthesis path. The two paths take separate options because the framing is different ("you ran out of budget" vs "the controller committed").
:temperature - Sampling temperature applied to all three LLM call sites (controller, answer rewrite, fallback synthesis). When omitted, the model's own default is used.
:controller_temperature - Override :temperature for the controller call only. Useful when you want a low temperature (e.g. 0.0-0.2) for tool routing decisions but a higher one for answer prose.
:answer_temperature - Override :temperature for the :answer_llm rewrite call only.
:fallback_temperature - Override :temperature for the fallback synthesis call only.