Behaviour every provider must implement.
A provider is a thin adapter between ExAthena.Request and a remote (or
local) inference endpoint. It is expected to:
- Normalise the request into the provider's native wire format.
- Perform the HTTP call (or SDK call, for Claude).
- Parse the response (or stream) back into an
ExAthena.Response/ExAthena.Streaming.Eventsequence. - Surface errors as
{:error, %ExAthena.Error{}}tuples using the canonical kinds.
Capabilities
Each provider declares its capabilities statically. The loop uses these to
decide the tool-call protocol and fallback strategy. See
ExAthena.Capabilities for the shape.
Summary
Callbacks
Static capability map for this provider.
Model-aware capability map. Receives the per-call opts (includes
:req_llm_provider_tag and :model) so the provider can resolve
the actual context window from a catalog (e.g. llm_db) and return
a precise max_tokens instead of a static default.
Perform a one-shot request and return the final response.
Stream a request; callback is invoked with each Streaming.Event. Must
still return {:ok, final_response} when the stream completes normally.
Callbacks
@callback capabilities() :: ExAthena.Capabilities.t()
Static capability map for this provider.
@callback capabilities(opts :: keyword()) :: ExAthena.Capabilities.t()
Model-aware capability map. Receives the per-call opts (includes
:req_llm_provider_tag and :model) so the provider can resolve
the actual context window from a catalog (e.g. llm_db) and return
a precise max_tokens instead of a static default.
When not implemented, the loop falls back to capabilities/0.
@callback query(ExAthena.Request.t(), opts :: keyword()) :: {:ok, ExAthena.Response.t()} | {:error, term()}
Perform a one-shot request and return the final response.
@callback stream( ExAthena.Request.t(), (ExAthena.Streaming.Event.t() -> term()), opts :: keyword() ) :: {:ok, ExAthena.Response.t()} | {:error, term()}
Stream a request; callback is invoked with each Streaming.Event. Must
still return {:ok, final_response} when the stream completes normally.