Runs an agentic tool-calling loop on top of LlmComposer.run_completion/3.

Where LlmComposer.simple_chat/2 and LlmComposer.run_completion/3 perform a single model turn, LlmComposer.Agent.run/3 automates the full cycle:

ask → model requests tool calls → execute them → feed the results back → repeat
      → until the model returns a final, tool-free answer.

The loop is pure orchestration over existing building blocks (LlmComposer.FunctionExecutor, LlmComposer.FunctionCallHelpers) and does not change any provider behaviour.

Streaming

When settings.stream_response is true, run/3 returns {:ok, stream} where stream is a lazy Enumerable of LlmComposer.StreamChunk. The stream carries only the final, tool-free answer (token-by-token :text_delta chunks) followed by a terminal :done chunk whose :usage and :cost_info hold the run totals and whose metadata.agent_result holds the full LlmComposer.Agent.Result. A hard failure (e.g. :max_iterations_reached) is delivered as a terminal :error chunk instead.

Intermediate progress — tool calls, per-iteration data and reasoning — is not placed on the answer stream. It is exposed via :telemetry (see below) so a UI can subscribe without having to filter the answer.

{:ok, stream} = LlmComposer.Agent.run(settings, "Weather in Paris?")

stream
|> Enum.reduce(nil, fn
  %LlmComposer.StreamChunk{type: :text_delta, text: t}, acc -> IO.write(t); acc
  %LlmComposer.StreamChunk{type: :done, metadata: %{agent_result: r}}, _ -> r
  _other, acc -> acc
end)

All providers support the streaming agent. Note that the native :ollama provider does not emit tool-call deltas; for tool-call streaming with Ollama use the :open_ai provider pointed at Ollama's OpenAI-compatible endpoint.

Example

defmodule MyTools do
  def calculator(%{"expression" => expression}) do
    {result, _binding} = Code.eval_string(expression)
    result
  end
end

calculator = %LlmComposer.Function{
  mf: {MyTools, :calculator},
  name: "calculator",
  description: "Evaluates a math expression",
  schema: %{
    "type" => "object",
    "properties" => %{"expression" => %{"type" => "string"}},
    "required" => ["expression"]
  }
}

settings = %LlmComposer.Settings{
  providers: [{LlmComposer.Providers.OpenAI, [model: "gpt-4.1-mini", functions: [calculator]]}],
  system_prompt: "You are a helpful assistant.",
  api_key: "sk-...",
  track_costs: true
}

{:ok, result} = LlmComposer.Agent.run(settings, "How much is (2 + 3) * 4?")

result.response.main_response.content
# => "The result is 20."
result.iterations
# => 2

Options

• :max_iterations — maximum number of model turns before giving up. Defaults to 10. When exceeded while the model still wants to call tools, run/3 returns {:error, :max_iterations_reached}.
• :functions — list of LlmComposer.Function.t() available to the model. Defaults to the :functions configured in the settings' provider options, so you usually do not need to set this explicitly.
• :tool_execution — :sequential (default) runs tool calls one by one; :parallel runs them concurrently with Task.async_stream/3 while preserving result order.
• :tool_timeout — per-task timeout (ms or :infinity) used in :parallel mode. Defaults to :infinity.

Tool errors

When a tool call fails (:function_not_found, invalid arguments, or an exception during execution) the loop does not abort. Instead it feeds an "Error: ..." string back to the model as the tool result, giving the model a chance to recover or explain the failure. A model or network error returned by the provider, by contrast, aborts the loop and is returned as {:error, reason}.

Telemetry

The loop emits the following :telemetry events (in both synchronous and streaming modes):

• [:llm_composer, :agent, :run, :start | :stop | :exception] — the whole run. The :stop event includes an :iterations measurement and a :status (:ok/:error/:halted) metadata key.

• [:llm_composer, :agent, :iteration, :stop] — one per model turn. Measurement :tool_call_count; metadata :iteration, :cost_info, :final.

• [:llm_composer, :agent, :tool, :start | :stop | :exception] — one per tool execution. Metadata :name, :arguments, :metadata, :id and (on stop) :status (:ok/:error).

• [:llm_composer, :agent, :reasoning, :delta] — streaming only: one per intermediate reasoning fragment. Metadata :iteration and :reasoning.

Pass telemetry_metadata: map() to run/3 and the map is merged into the metadata of every event above, alongside an auto-generated :run_id. This lets a handler scope itself to a single run and, for example, broadcast to Phoenix.PubSub or send/2 a pid:

{:ok, _} = LlmComposer.Agent.run(settings, prompt, telemetry_metadata: %{conversation_id: cid})

:telemetry.attach("agent-ui", [:llm_composer, :agent, :tool, :start], fn
  _event, _meas, %{conversation_id: ^cid, name: name, arguments: args}, _cfg ->
    Phoenix.PubSub.broadcast(MyApp.PubSub, "conv:#{cid}", {:tool_call, name, args})
end, nil)