RAG — retrieval inside a council

Council members ground their answers in a corpus by calling a CouncilEx.Tool mid-completion. There is no separate "retriever" behaviour and no new abstraction. RAG reuses the existing tool surface plus two phase-1 additions:

• Council-level tools: DynamicCouncil.add_council_tool/2 attaches a tool to every member of a council, including the chair. Useful for a shared knowledge base every member should be able to query.
• CouncilEx.Tools.InMemoryDocs: pure-Elixir BM25 retrieval tool baked from a compile-time corpus. Zero deps. Reference impl + the default for examples and tests.

Deterministic, model-independent grounding (pre-injected context that members do not have to opt into) is deferred.

1. Why "RAG via tools"?

A retrieval tool keeps the existing tool-loop honest:

• Same telemetry ([:council_ex, :tool, :execute, :*]).
• Same parallel/timeout knobs.
• Same JSON / verbose tracer output.
• Composes with structured-output, streaming, sub-councils.

The trade-off is that retrieval is agentic: the model decides whether and what to query. Strong models (Sonnet/4o-class) reliably issue tool calls when prompts hint at policy/legal/contextual lookup; weaker models sometimes skip the tool. If guaranteed grounding is required, the deferred phase-2 pre-injection retriever is the right abstraction. Until then, pick capable models and write tool descriptions that name the trigger conditions explicitly.

2. Council-level vs member-level tools

Refs are the same as anywhere else in CouncilEx: a registered tool name ("search_docs") or a tool module atom (MyApp.Tools.SearchDocs).

When both surfaces are in play, council tools are merged into every member's resolved tool list before member-level tools, with module-level dedup. Two refs that resolve to the same module collapse into one entry, so a council ref "search_docs" and a member ref MyApp.Tools.SearchDocs produce a single tool, not two.

DynamicCouncil.new("policy-debate")
|> DynamicCouncil.set_default_profile("openrouter_default")
|> DynamicCouncil.add_council_tool(MyApp.Tools.PolicyDocs)         # all members
|> DynamicCouncil.add_member(%{
     id: "support",
     system_prompt: "Answer with policy citations."
   })
|> DynamicCouncil.add_member(%{
     id: "legal",
     # Legal sees PolicyDocs (council-wide) + LegalPrecedents (its own).
     tools: [MyApp.Tools.LegalPrecedents],
     system_prompt: "Flag liability/regulatory exposure."
   })
|> DynamicCouncil.add_round(:independent_analysis)
|> DynamicCouncil.set_chair(%{id: "chair", system_prompt: "Synthesize."})

Sub-council members are excluded from the merge. A member declared with :sub_council runs a nested council; the parent's council tools do not leak into the inner run. If the inner council needs the same toolset, attach it there too.

JSON ser/de round-trips council tools the same way as member tools: strings stay strings; module atoms are emitted as %{"module" => "..."} markers and rehydrated via String.to_existing_atom/1.

Validation surfaces unknown council tool refs at ["tools", N] with code :unknown (same shape as member-level tool errors). Always validate before persisting / running:

:ok = CouncilEx.DynamicCouncil.validate!(council)

3. CouncilEx.Tools.InMemoryDocs

Build a tool from a compile-time corpus. BM25 ranking, pure Elixir, zero deps. Designed for examples, tests, and small shared knowledge bases. Production users with real corpora should write their own CouncilEx.Tool that wraps a vector DB or search service.

defmodule MyApp.Tools.PolicyDocs do
  use CouncilEx.Tools.InMemoryDocs,
    name: "search_policy",
    description: """
    Search the company policy corpus. Use this whenever the question
    touches refund, return, warranty, or shipping policy. Pass the
    user's intent as the query string.
    """,
    docs: [
      %{
        text: "Refund policy: full refund within 30 days of delivery …",
        meta: %{section: "refunds", source: "policy-v3.2"}
      },
      %{
        text: "Warranty policy: standard products carry a 12-month …",
        meta: %{section: "warranty", source: "policy-v3.2"}
      }
      # plain strings work too; auto-wrapped to %{text: …, meta: %{}}
    ],
    top_k: 4
end

use expands at compile time, computes the BM25 index, and embeds both the corpus and the index in the generated module. The result is a regular CouncilEx.Tool with name/0, description/0, parameters_schema/0, and execute/1. Drop it in anywhere a tool ref is accepted.

Tool parameters

Tool result

A list of maps, ranked by descending score, zero-score docs filtered:

[
  %{text: "...", meta: %{section: "warranty", ...}, score: 1.83},
  %{text: "...", meta: %{section: "refunds", ...},  score: 1.21}
]

The provider serializes this back into the tool message the model sees on its next turn (string for binary results, JSON-encoded for maps and lists; see the OpenAI / Anthropic / Gemini adapters' continue_with_tool_results/2).

BM25 details

Standard Robertson/Spärck-Jones BM25 with k1 = 1.2, b = 0.75. Token filter: lowercase, split on \W+, drop tokens shorter than two characters and a small English stop-word list. Knobs are intentionally fixed: if you need to tune them, you should be writing a real retrieval tool, not extending this helper.

Public surface (useful in tests + ad-hoc usage):

CouncilEx.Tools.InMemoryDocs.build_index(docs)        # %{docs, tf, df, dl, avgdl, n}
CouncilEx.Tools.InMemoryDocs.search(index, q, top_k)  # [%{text, meta, score}]
CouncilEx.Tools.InMemoryDocs.tokenize(text)           # [token]
CouncilEx.Tools.InMemoryDocs.normalize_doc(item)      # %{text, meta}

4. End-to-end example

examples/rag_via_tools.exs shows both surfaces against a real provider:

• A shared PolicyDocs corpus attached at council level.
• A LegalPrecedents corpus attached only to the legal member.
• Three-member dynamic council (support / legal / chair) running through :independent_analysis → synthesis over OpenRouter.

Run:

OPENROUTER_API_KEY=sk-or-… mix run examples/rag_via_tools.exs

RAG_EXAMPLE_DRYRUN=1 short-circuits before the model run, used by test/council_ex/examples/rag_via_tools_test.exs to keep the example in sync with the library without burning API credits.

VERBOSE=1 adds the per-call tracer (member start/stop, tool-call args and results). Tool calls flow through the same telemetry surface as any other tool: [:council_ex, :tool, :execute, :*] events fire identically.

5. Tips and gotchas

• Write tool descriptions that name the trigger. Models call tools more reliably when the description names the conditions ("Use this whenever the question touches refund, warranty, …") than when it states the capability abstractly.
• Keep top_k small. Default 4. The full passage list is serialized into the next assistant turn; over-fetching wastes context.
• Dedup is module-level, not ref-level. A council ref "foo" and a member ref MyApp.Tools.Foo collapse to one entry, but only after registry resolution. Atom-only refs that point to different modules with the same name are not deduped.
• Sub-council members do not inherit council tools. Attach tools inside the sub-council if the inner debate needs them.
• Mock provider can drive RAG too. Standard Mock-provider scripts can simulate tool calls + results. See the test/council_ex/dynamic_council_test.exs "council-level tools" describe block for the wiring.