Models, budgets and limits

ExAgent uses official stock ReqLLM as its general backend. Custom adapters implement ExAgent.Model; the Test adapter runs locally. A model being present in a catalogue is distinct from its tools, output and streaming profile being qualified for ExAgent.

Resolve a text model

The application supplies credentials. This example resolves configuration; calling ExAgent.run/3 with it makes a real provider request:

{:ok, model} = ExAgent.Model.resolve({:openai, id: "gpt-4o-mini"},
  api_key: System.fetch_env!("OPENAI_API_KEY")
)
agent = ExAgent.new(model: model, instructions: "Be concise.")

Use instance credentials for explicit ownership. ReqLLM's startup environment loading is described in Getting started. Never place a key in a prompt or persist the complete live model.

Catalogue updates can remove a model or change its prices, limits and capabilities. ExAgent 2.2.2 requires ReqLLM 1.29 and its newer LLMDB catalogue; the tested lock uses LLMDB 2026.10.6. Some former OpenRouter entries, including unbiased/pareto and inclusionai/ling-3.0-flash-fin, are no longer present. Resolving an absent ID can still succeed as an unverified model without catalogue metadata. Do not interpret that success as tool qualification or known pricing.

Applications that need stable metadata can pass an explicit model map to ExAgent.Model.resolve/2 or ExAgent.Models.ReqLLM.new/1, with the provider, exact id, verified capabilities, limits, cost and wire protocol they rely on. Maintain those facts against the actual provider; ExAgent does not restore retired catalogue entries or invent prices. Review a model change before resuming persisted work, and keep continuation mismatch checks intact.

ReqLLM 1.29 can merge provider_options from its application configuration or the resolved model's extra metadata. ExAgent rejects non-empty defaults with {:unsupported_options, :provider_defaults} before IO and continuation binding: these options would otherwise bypass the instance allowlist and persisted route. Remove those defaults and pass supported options explicitly as ExAgent.Models.ReqLLM.new(provider_options: ...). Empty defaults are accepted.

Configure Chat tools and streaming

The explicit profile binds the protocol, tools and reasoning capability:

model = ExAgent.Models.ReqLLM.new(
  model: %{
    provider: :openai,
    id: "gpt-4o-mini",
    capabilities: %{tools: %{enabled: true}, reasoning: %{enabled: false}},
    extra: %{wire: %{protocol: "openai_chat"}}
  },
  api_key: System.fetch_env!("OPENAI_API_KEY"),
  tool_profile: :chat_tools_v1
)

These declarations configure admission; they do not prove the endpoint behaves correctly. Qualify the exact gateway/model combination you deploy. The supported profiles describe the available tool/output combinations. Unrepresented reasoning, modality and continuation combinations reject explicitly.

Native output is separate: output_profile: :chat_json_schema_v1 plus agent output_mode: :native. See Tools and output.

Route OpenRouter Chat tools explicitly

Use tool_profile: :openrouter_chat_tools_v1 for a qualified OpenRouter Chat endpoint. The host supplies the exact model ID and truthful capabilities:

model = ExAgent.Models.ReqLLM.new(
  model: %{
    provider: :openrouter,
    id: qualified_model_id,
    capabilities: %{tools: %{enabled: true}, reasoning: %{enabled: false}},
    extra: %{wire: %{protocol: "openai_chat"}}
  },
  api_key: System.fetch_env!("OPENROUTER_API_KEY"),
  tool_profile: :openrouter_chat_tools_v1,
  provider_options: [
    openrouter_provider: %{only: [qualified_provider_id], allow_fallbacks: false}
  ]
)

Routing admits order, only, ignore, allow_fallbacks and require_parameters. Provider lists contain 1–32 bounded identifiers. Optional app_title and app_referer are attribution; other wire settings remain closed. The mandatory tool argument envelope and validation still apply.

For a reasoning-capable model, reasoning_mode: :none additionally requires declared support for effort "none" and disabling thinking. It sends one max_tokens budget. Declaring those capabilities does not establish real provider compatibility. This tools profile does not enable native JSON output.

The route and reasoning mode are bound before the first request and in tool history. Changing them requires a fresh conversation; an existing approval or uncertain request cannot resume through another route. Earlier unbound OpenRouter history is rejected. See Migration for continuation version 4 and the unaffected OpenAI history versions.

Send images and documents

ExAgent.run/3 also accepts a list of content parts instead of a string:

ExAgent.run(agent, [
  %{type: "text", text: "Summarize the report and describe the chart."},
  %{type: "image", data: Base.encode64(png), media_type: "image/png"},
  %{type: "file", data: Base.encode64(pdf), media_type: "application/pdf",
    filename: "report.pdf"}
])

Parts are text, image_url (http, https or data URL), image and file (base64 data). The list is checked and stored with string keys before any request; an invalid part fails with {:invalid_prompt_part, index} and an empty list with :invalid_prompt. The model then accepts or rejects each modality before IO: images need :image in the model's input modalities, and PDFs need :pdf and a provider that sends documents (OpenRouter, Anthropic or Google; stock OpenAI Chat accepts only images). Base64 content counts towards max_history_bytes, so raise it for large files.

Cache stable prompt prefixes

prompt_cache lowers the cost of long instructions and growing tool histories without changing what is sent:

model = ExAgent.Models.ReqLLM.new(
  model: openrouter_spec,
  api_key: System.fetch_env!("OPENROUTER_API_KEY"),
  tool_profile: :openrouter_chat_tools_v1,
  prompt_cache: [ttl: "5m"]
)

On OpenRouter it marks the last system message and the newest message with a cache_control breakpoint, which Anthropic and Gemini routes use; ttl is "5m" or "1h". OpenAI caches long prefixes automatically, and prompt_cache: [key: "tenant:42"] sends a stable prompt_cache_key to improve hits. Other providers reject the option before IO. Cache reads and writes show up in result.usage.details when the provider reports them.

Retry and fall back to another model

ExAgent.Models.Fallback retries transient provider failures and then tries the next model:

primary = ExAgent.Models.ReqLLM.new(primary_options ++ [foreign_history: :portable])
backup = ExAgent.Models.ReqLLM.new(backup_options ++ [foreign_history: :portable])

model = ExAgent.Models.Fallback.new(models: [primary, backup], max_retries: 2)
agent = ExAgent.new(model: model, tools: tools)

Only failures that returned no output are retried: HTTP 408, 429, 500, 502, 503, 504 and 529 by default (:retry_statuses). A gateway may still bill an upstream attempt behind a 502 or 504. Connection failures and total timeouts may already have been processed and billed, so they need retry_transport: true. Backoff is exponential with jitter. Each attempt keeps ModelSettings.timeout as its receive timeout, and the run deadline bounds all attempts together. A stream is retried only if it fails before its first event.

A retried request is still one admitted request: request_limit and result.request_count count model requests, not HTTP attempts. Each retry emits [:exagent, :model, :retry] telemetry. An explicit estimate_cost callback receives the Fallback model rather than the one that answered; price by the response's model_name or rely on usage the models report.

Switching models mid-conversation sends one model's responses to another. Models.ReqLLM rejects that by default. With foreign_history: :portable it sends history it cannot replay natively (another model's responses, or its own from another route or reasoning mode) as plain text and tool calls, without their reasoning, signatures or provider metadata, instead of rejecting it. Tool calls keep ExAgent's argument codec, so tool history stays exact. A conversation that only uses one model and route is sent exactly as before. Fallback skips a candidate whose optional preflight reports a history target mismatch, without IO or retry telemetry. A persisted backup conversation can therefore continue with that compatible model. If all candidates mismatch, the request fails; malformed history and other configuration errors remain terminal.

Bound requests and tool calls

agent = ExAgent.new(
  model: "test",
  usage_limits: %ExAgent.UsageLimits{request_limit: 3, tool_calls_limit: 2}
)
{:ok, result} = ExAgent.run(agent, "Hello!")
result.request_count
# => 1

Request and tool counters are host observations. Limits apply before admission of the next operation. Scoped child runs inherit ancestor authority and budgets; a child cannot enlarge the parent's allowance. Parent usage is inclusive of its children, so adding every span's totals would count delegated work twice.

For token/cost dimensions, read ExAgent.UsageLimits and ExAgent.CostGuard. Strict metric accounting requires a finite request limit. Provider-reported or normalized tokens and estimated cost have quality/status fields. Unknown usage or cost is not zero, and an estimated threshold is not a guaranteed invoice ceiling.

Finish at the step limit

By default a run that reaches max_steps fails with :max_steps_exceeded. With on_max_steps: :final, the last allowed step asks for an answer instead:

agent = ExAgent.new(model: model, tools: tools, max_steps: 8, on_max_steps: :final)

In step max_steps the function tools stay declared but cannot be called (the model gets tool_choice: none, or the output tool for structured output), and that request alone ends with a user instruction saying the limit was reached. The instruction is not added to the conversation history. If the model still calls a function tool, the call is not executed (it gets a not_executed return) and the run fails with :max_steps_exceeded. The run never makes more than max_steps model requests.

Separate the timeout boundaries

Stopping an owned task closes framework resources. It cannot roll back external IO or guarantee cancellation of arbitrary application callbacks. Host retention limits apply after stock decoding; no upstream hard RAM bound before decoding is promised. Migration documents option precedence and failure shapes.

An incomplete response stays in RunError.partial.pending_response, outside executable history. Its duplicate in RunError.reason may carry an explicit payload_omitted marker when that error copy would exceed 4 KiB. Read the cause from the bounded RequestError and diagnostic partial data from RunError; neither authorizes tools or proves that a turn completed. Arbitrary oversized error control still fails the original retention limit.

API: ExAgent.Models.ReqLLM, ExAgent.Models.Fallback, ExAgent.Model, ExAgent.ModelSettings.