Behaviour for LLM providers.
A provider is an interchangeable service (OpenAI, Gemini, a self-hosted
vLLM container, ...)
implementing a shared capability. Each provider is a struct module that
declares @behaviour ExAgent.Provider and implements chat/3 (required)
and optionally upload/4 and stream/3. The provider struct carries its own
config (api key, model, cached Req client) and is passed as the first argument.
Failures are returned as {:error, %ExAgent.Error{}}.
This module also acts as the dispatcher: it forwards to the callback module resolved from the provider struct.
Implementations
What the library requires of a provider struct
Nothing beyond being a struct. The dispatcher resolves the callback module from
provider.__struct__ and never inspects fields. Optional conveniences:
- a
:toolsfield - the agent populates it before each turn so the service can read it back; a provider with no tool support simply omits it - a
:system_promptfield - read by that provider's own service, not by the library
Everything vendor-specific - accepted modalities, inline size ceilings, content part shapes, embedding task vocabularies - belongs to the provider and its service, so a new provider is free to disagree with every one that ships here.
Extensibility
To add a new provider, define a struct and implement this behaviour.
ExAgent.Error.from_result/2 handles the status classification:
defmodule MyApp.Providers.CustomLLM do
@behaviour ExAgent.Provider
defstruct [:api_key, :model, :base_url, :system_prompt, :req, tools: []]
@impl true
def chat(provider, messages, opts) do
Req.post(provider.req, url: "/chat", json: build_body(messages, opts))
|> ExAgent.Error.from_result(__MODULE__)
|> case do
{:ok, body} -> parse_response(body)
{:error, _error} = failure -> failure
end
end
endSummary
Callbacks
Sends a list of messages to the LLM and returns the assistant's response.
Generates embedding vectors for a list of inputs.
Returns the embedding task atoms this provider accepts.
Reorders documents by relevance to query.
Streams the assistant's response as a lazy enumerable of text chunks.
Returns the attachment modalities this provider instance accepts.
Whether this provider instance can constrain output to a JSON Schema.
Uploads binary file data to the provider and returns a file reference.
Functions
Dispatches a chat request to the provider's implementation.
Dispatches an embedding request to the provider's implementation.
Returns the embedding task atoms provider accepts, or [] if it has none.
Dispatches a rerank request to the provider's implementation.
Dispatches a streaming chat request to the provider's implementation.
Returns the attachment modalities the provider accepts.
Dispatches a file upload to the provider's implementation.
Callbacks
@callback chat(provider :: struct(), [ExAgent.Message.t()], keyword()) :: {:ok, ExAgent.Response.t()} | {:tool_calls, [map()]} | {:tool_call, String.t(), map()} | {:error, ExAgent.Error.t()}
Sends a list of messages to the LLM and returns the assistant's response.
Returns {:ok, %ExAgent.Response{}} for a regular response, {:tool_calls, calls}
when the LLM wants to invoke tools, or {:error, %ExAgent.Error{}} on failure.
Each call is a map with "name", "args", and - where the provider issues
one - "id". Models request several tools in a single turn, so this is a
list; returning only the first left the model believing tools had run that
never did.
{:tool_call, name, args} remains accepted for a single call, so providers
written against the older contract keep working.
@callback embed(provider :: struct(), [ExAgent.Embeddings.input()], keyword()) :: {:ok, ExAgent.Embeddings.t()} | {:error, ExAgent.Error.t()}
Generates embedding vectors for a list of inputs.
Optional - providers without an embeddings endpoint omit this callback.
@callback embedding_tasks(provider :: struct()) :: [ExAgent.Embeddings.task()]
Returns the embedding task atoms this provider accepts.
Optional - providers with no task field return [].
There is no shared vocabulary: Gemini's taskType is a closed enum of eight
values, Jina v5 has four names plus a separate prompt_name, and OpenAI has no
task at all. A provider that translated a common set into its own would have to
either drop distinctions its model makes or invent ones it does not.
@callback rerank( provider :: struct(), query :: String.t(), documents :: [String.t()], keyword() ) :: {:ok, ExAgent.Reranking.t()} | {:error, ExAgent.Error.t()}
Reorders documents by relevance to query.
Optional - providers without a reranking endpoint omit this callback.
A reranker is a cross-encoder: it reads the query and one document together rather than comparing independently-computed vectors, which is why it is accurate enough to order a shortlist and too slow to search a corpus.
@callback stream(provider :: struct(), [ExAgent.Message.t()], keyword()) :: Enumerable.t()
Streams the assistant's response as a lazy enumerable of text chunks.
Optional - providers without streaming support omit this callback.
@callback supported_modalities(provider :: struct()) :: [ExAgent.Source.modality()]
Returns the attachment modalities this provider instance accepts.
Optional - providers that omit it are treated as text-only, so an attachment they cannot handle fails loudly instead of being dropped on the floor.
Takes the provider struct, not the module, because a provider pointed at a configurable backend declares its modalities per instance.
Whether this provider instance can constrain output to a JSON Schema.
Optional - omitting it means no, so a provider that has not opted in refuses a
:schema rather than quietly ignoring it and returning prose.
@callback upload(provider :: struct(), binary(), String.t(), keyword()) :: {:ok, ExAgent.FileRef.t()} | {:error, ExAgent.Error.t()}
Uploads binary file data to the provider and returns a file reference.
Optional - providers without a files API (e.g. ExAgent.Providers.OpenAICompatible)
omit this callback.
Functions
@spec chat(struct(), [ExAgent.Message.t()], keyword()) :: {:ok, ExAgent.Response.t()} | {:tool_calls, [map()]} | {:tool_call, String.t(), map()} | {:error, ExAgent.Error.t()}
Dispatches a chat request to the provider's implementation.
Returns {:error, %ExAgent.Error{type: :unsupported}} if any attachment's
modality is not in the provider's supported_modalities/1.
@spec embed(struct(), [ExAgent.Embeddings.input()], keyword()) :: {:ok, ExAgent.Embeddings.t()} | {:error, ExAgent.Error.t()}
Dispatches an embedding request to the provider's implementation.
Returns {:error, %ExAgent.Error{type: :unsupported}} if the provider does not
implement embed/3. Follows the upload/4 tuple convention rather than
stream/3's raise: embed/3 already returns a result tuple for every other
failure, and raising for just one of them would force callers to write both a
case and a try.
@spec embedding_tasks(struct()) :: [ExAgent.Embeddings.task()]
Returns the embedding task atoms provider accepts, or [] if it has none.
@spec rerank(struct(), String.t(), [String.t()], keyword()) :: {:ok, ExAgent.Reranking.t()} | {:error, ExAgent.Error.t()}
Dispatches a rerank request to the provider's implementation.
Returns {:error, %ExAgent.Error{type: :unsupported}} if the provider does not
implement rerank/4.
@spec stream(struct(), [ExAgent.Message.t()], keyword()) :: Enumerable.t()
Dispatches a streaming chat request to the provider's implementation.
Returns a lazy enumerable of text chunks. Raises ExAgent.Error with
type: :unsupported if the provider does not implement stream/3, or if any
attachment's modality is unsupported - a lazy enumerable has nowhere to carry
an error tuple at construction time.
@spec supported_modalities(struct()) :: [ExAgent.Source.modality()]
Returns the attachment modalities the provider accepts.
Defaults to [:text] for providers that do not implement
supported_modalities/1.
@spec upload(struct(), binary(), String.t(), keyword()) :: {:ok, ExAgent.FileRef.t()} | {:error, ExAgent.Error.t()}
Dispatches a file upload to the provider's implementation.
Returns {:error, %ExAgent.Error{type: :unsupported}} if the provider does not
implement upload/4.