A scripted model client for tests: no network, no API key, no Normandy.Agents.Model implementation of your own.

new/2 takes the replies in call order. Each request takes the next one, whichever process makes it (Normandy.Agents.BaseAgent.run/3, Normandy.Agents.BaseAgent.stream/3, a Normandy.Agents.Turn.Session's server, a Normandy.Coordination.AgentProcess), as long as it runs on the node that built the client. requests/1 returns every request received so far, oldest first, as Normandy.LLM.Request structs.

client =
  Normandy.Test.FakeClient.new([
    {:tool_calls, [{"weather", %{"city" => "Oslo"}}]},
    {:text, "Cold."}
  ])

agent =
  Normandy.Agents.BaseAgent.init(client: client, model: "claude-opus-5-5")
  |> Normandy.Agents.BaseAgent.register_tool(%MyApp.WeatherTool{})

{:ok, _agent, %{chat_message: "Cold."}} =
  Normandy.Agents.BaseAgent.run(agent, "Weather in Oslo?")

[_tool_round, final_request] = Normandy.Test.FakeClient.requests(client)

Script items

• {:text, text} - the answer text. text is a binary, or a list of binaries that stream/3 sends as one text delta each (converse/3 joins them). For an output schema other than the chat schema, the text is parsed into it as JSON, as the Claudio adapter parses a model's text (the client's :on_parse_failure applies).
• {:structured, map} - a structured answer: map encoded as JSON, then parsed into the request's response model, or sent as the text of a tool loop's final answer.
• {:tool_calls, calls} - one or more tool calls. Each call is {name, input} (its id is "toolu_fake_<request>_<position>", both counted from 0) or a %Normandy.Components.ToolCall{} used as given. Only a request that lists tools takes them: scripted for one that lists none, they raise.
• {:error, %Normandy.LLMError{}} - the call fails with that error; its retryable? decides whether a configured retry: resends the request.
• a 1-arity function - called with the %Normandy.LLM.Request{}; returns one of the items above.

:text, :structured and :tool_calls take a keyword list as a third element:

• :stop_reason - the response's stop reason (default :tool_use for tool calls, :end_turn otherwise). :max_tokens and :refusal make Normandy return {:error, %Normandy.LLMError{type: :max_tokens | :refusal}}.
• :usage - the usage map the response reports (default nil, no usage), for example %{input_tokens: 12, output_tokens: 3}.
• :stream_error_after - streaming only: the stream ends in a transport error after this many content deltas (at its end when the reply has fewer).

new/2 checks every item and raises ArgumentError naming the first bad one.

The end of the script

A request past the last item raises Normandy.Test.FakeClient.ScriptExhaustedError: run/3 and stream/3 raise it, and a Normandy.Agents.Turn.Server replies {:error, %Normandy.TurnError{type: :task_down}} whose reason is the exception. With retry: configured, BaseAgent retries a raised call, so each retry takes a request index too. remaining/1 tells how many items no request has taken.

The requests live in an ETS table owned by the process that called new/2: build the client in the test process, not in setup_all or a task that exits before the test ends.