ExAgent.Models.ReqLLM (ExAgent v2.2.0)

View Source

Model adapter over ReqLLM's public one-interaction APIs.

:model accepts a catalogue string or an explicit model spec. Supply :api_key explicitly; :base_url selects an instance/gateway. Anthropic also accepts explicit :auth_token (Bearer), which takes precedence over :api_key through stock public OAuth options without enabling subscription behavior. Neither mode reads credentials from the environment. Model capabilities for tools and image input must be declared in the spec when absent from the catalogue.

:http_options currently accepts :adapter (trusted transport injection) and :receive_timeout. :provider_options accepts Anthropic version/top-k, stop sequences and Responses :store (use false for stateless continuation). The explicit :openrouter_chat_tools_v1 profile additionally permits bounded :openrouter_provider routing and :app_title/:app_referer attribution. Other options fail explicitly, never override auth, tools, history, model or retries. Native JSON output requires the separate explicit output_profile: :chat_json_schema_v1 alongside tool_profile: :chat_tools_v1. It sends a reference-free object schema via the public response_format option, with strict: false, preserving optional fields and inert defaults. Set the agent's output_mode: :native with an Ecto module; local schema validation and the actual changeset are authoritative, with counted corrective retries. Streaming is limited to the explicit Chat tool profiles below and rejects HTTP :adapter injection (stock streaming uses its own public transport path).

:prompt_cache (default nil; false also disables it) enables provider prompt caching without changing history, tools or continuation binding. On OpenRouter, true or [ttl: "5m" | "1h"] adds cache_control breakpoints to the last system message and the newest message (used by Anthropic and Gemini routes). OpenAI caches automatically; [key: key] sends a stable prompt_cache_key to improve cache hits. Other providers reject the option before IO. Cache reads and writes appear in the usage details when the provider reports them.

Public ReqLLM Response metrics retain this adapter's conservative normalized accounting contract. ReqLLM 1.26 adds usage-presence metadata, but its complete provider/surface semantics are not qualified as observed host accounting. Metrics are exposed as Usage.accounting quality "normalized", including zero; unavailable metrics remain nil. Public USD totals are estimates converted once to cents. Strict metric limits reject this declared normalized contract before IO. Opt in to UsageLimits.accounting: :estimated with a finite effective request limit for retrospective thresholds; neither mode promises an invoice or monetary cap.

Tools and Ecto tool output require tool_profile: :chat_tools_v1 (OpenAI) or :openrouter_chat_tools_v1 (OpenRouter), explicit extra: %{wire: %{protocol: "openai_chat"}} model metadata, tools enabled and reasoning disabled, or explicit reasoning_mode: :none with truthful reasoning enabled, effort supported with "none" among its values, and thinking.disable_supported=true. None mode sends public reasoning_effort:none, requires nil temperature, and maps the sole ModelSettings.max_tokens authority (1..4096, default4096) to max_completion_tokens for OpenAI, or canonical max_tokens for OpenRouter, without duplicate budget keys. It rejects exposed reasoning and does not qualify reasoning continuation. This non-strict profile validates a mandatory wire envelope and a reference-free logical schema subset locally. Defaults remain inert and optional fields stay optional. Strict tools and additional provider options reject. Callers/hooks use logical objects; continuation2 and call metadata identify exagent.arguments/1. OpenAI none writes continuation3 with an explicit reasoning_mode binding and rejects legacy/different-mode response history. Persisted none execution additionally exposes a static, nonsecret Model binding checked before and after app-codec load, even before a first response exists. OpenRouter writes continuation4 with bounded routing and disabled/none mode; both modes bind routing statically and reject changed or legacy routed history. Pre-envelope call history rejects, without repair or migration by guessing. Other profiles reject tool requests because stock normalization can turn non-object arguments into {}. Select the explicit compatible profile described in the model and migration guides. Custom Model and Test remain independent. Anthropic thinking-enabled requests and response continuation are unsupported. ReqLLM 1.26 preserves opaque redacted provider blocks; this adapter rejects them because its canonical continuation contract cannot round-trip them; accepting the :thinking option for validation does not enable that route.

Receive timeout precedence is request ModelSettings.timeout, instance http_options[:receive_timeout], then 60 seconds. This is not a total deadline. Stock ReqLLM also uses this value as its pool-checkout default; the adapter does not expose a separate pool option. :total_timeout is an optional positive millisecond budget for this single buffered ReqLLM operation. Nil inherits the public ReqLLM application default (normally infinity). A host guardian applies that deadline and owns one linked worker; ReqLLM's internal total timeout is disabled to avoid orphaned nolink HTTP work on caller death. Success, exception, timeout and caller death terminate the owned worker. This is distinct from the run deadline, does not undo effects and does not guarantee cleanup of arbitrary external callback descendants. Instructions come from the supplied canonical/projected message history; ModelRequestParameters.instructions is not independently prepended. Qualified function tools add one backend-only system message explaining their mandatory arguments envelope. Caller messages and canonical history stay unchanged; this guidance does not repair or admit invalid tool arguments.

Streaming is lazy, consumes ReqLLM.StreamResponse.process_stream/2 once, and shares buffered schema/history/response validation. Deltas are provisional; only a validated complete terminal authorizes tools or final output. Each new enumeration starts one new interaction. Halt, exception, deadline and owner death close the public backend response and terminate owned work.

The qualified stream profile admits at most 4096 public chunks, 65,536 bytes per chunk and 1,048,576 bytes summed across chunks, measured after decode using :erlang.external_size/1 (serialized representation, not heap size). The rejecting chunk has already been allocated upstream. The bridge holds one unacknowledged delta; the upstream collector retains O(response) data, and upstream queues/accumulators can run ahead. This is not end-to-end backpressure or a hard RAM/predecode limit. Canonical input/history and caller-retained events are separate; scope request/concurrency/deadline limits remain app-configured.

Streaming requests default to max_tokens: 4096 when unset and reject explicit values outside 1..4096 before IO. Streaming total_timeout: nil means 60,000ms, unlike buffered's inherited default; explicit values must be in 1..300,000ms for this qualified profile. Slow-consumer waiting counts toward that deadline. Completion is timestamped before delivery, preserving timely queued results and rejecting late completion even if the guardian timer was delayed. Cancellation cannot undo effects from previously completed interactions. Errors without a public usage snapshot remain unavailable; this profile does not certify a live provider or all ReqLLM models.

Native stream objects exposed by stock become canonical JSON Text history; original JSON whitespace/key ordering is not promised. Any publicly exposed refusal rejects. Stock Chat 1.26 can discard a wire refusal field alongside valid JSON content, so that content can still produce a locally valid output; refusal detection is not a wire-fidelity guarantee. A refusal without valid output, malformed/non-object JSON and incomplete terminals never succeed. Native output with qualified function tools is supported; tools retain their envelope, authority and effect accounting. Provider strict enforcement and parallel_tool_calls: false are not promised. Verify the exact endpoint/model configuration before relying on its native output behavior.

Summary

Types

t()

@type t() :: %ExAgent.Models.ReqLLM{
  api_key: String.t() | nil,
  auth_token: String.t() | nil,
  base_url: String.t() | nil,
  foreign_history: :reject | :portable,
  http_options: keyword(),
  model: String.t() | map() | tuple(),
  output_profile: :chat_json_schema_v1 | nil,
  prompt_cache: boolean() | keyword() | nil,
  provider_options: keyword(),
  reasoning_mode: :none | nil,
  tool_profile: :chat_tools_v1 | :openrouter_chat_tools_v1 | nil,
  total_timeout: pos_integer() | nil
}

Functions

new(opts)

@spec new(keyword()) :: t()