Behaviour and shared logic for LLM providers.

A provider represents a specific LLM service — its endpoint, authentication mechanism, and any service-specific adaptations. It is the authenticated HTTP layer: it knows where to send requests and how to authenticate them, but not what the request body should contain (that's the dialect's job).

Omni separates providers from dialects because the mapping is many-to-one. There are ~4–5 wire formats but ~20–30 services. Groq, Together, Fireworks, and OpenRouter all speak the OpenAI Chat Completions format — their request bodies and streaming events are identical. Only endpoint, authentication, and the occasional quirk differ. A provider captures those differences; a dialect (see Omni.Dialect) handles the wire format shared across many providers.

Most providers speak a single dialect, but multi-model gateways (like OpenCode Zen) route to different upstream APIs depending on the model. These providers omit the :dialect option — the dialect is resolved per-model from the catalog data at load time. See "Multi-dialect providers" below.

Defining a provider

Use use Omni.Provider with a required :id option — the provider's canonical id, its models.dev catalog key snake_cased as an atom — and typically a :dialect. The only required callback is config/0; models load automatically from the configured model source (see Omni.Source) under the provider's id:

defmodule MyApp.Providers.Mistral do
  use Omni.Provider,
    id: :mistral,
    dialect: Omni.Dialects.OpenAICompletions

  @impl true
  def config do
    %{
      base_url: "https://api.mistral.ai",
      api_key: {:system, "MISTRAL_API_KEY"}
    }
  end
end

The use macro generates id/0 and dialect/0 accessors (dialect is nil when omitted) and default implementations for all optional callbacks. Override only what your provider needs — most providers are just config/0.

Override models/0 to adjust what the source returns, or to skip catalog loading entirely — see models/0. The decision rule: implement an Omni.Source when you're replacing the data supply (reusable across providers, user-configurable); override models/0 when one provider needs local adjustments or its own loading path.

The request pipeline

Understanding the request pipeline clarifies when each callback runs and why. Internally, Omni orchestrates the flow by calling dialect and provider callbacks via the module references on the %Model{} struct:

# 1. Dialect builds the request body from Omni types
body = dialect.handle_body(model, context, opts)

# 2. Provider adjusts the body for service-specific quirks
body = provider.modify_body(body, context, opts)

# 3. Dialect determines the URL path for this API
path = dialect.handle_path(model, opts)

# 4. Provider builds the full URL (base_url + path)
url = provider.build_url(path, opts)

# 5. Provider adds authentication to the request
{:ok, req} = provider.authenticate(req, opts)

On the response side, streaming events flow through a similar pipeline:

# 1. Dialect parses raw SSE JSON into normalized delta tuples
deltas = dialect.handle_event(raw_event)

# 2. Provider adjusts deltas for service-specific data
deltas = provider.modify_events(deltas, raw_event)

The dialect does the heavy lifting (full type translation); the provider makes small, targeted adjustments. Most providers don't override modify_body/3 or modify_events/2 at all.

Callbacks

Authentication

The default authenticate/2 resolves opts.api_key via resolve_auth/1 and sets the appropriate header. When no :auth_header is configured, it sends a Bearer token on the "authorization" header (the most common scheme). When a custom :auth_header is set in config/0 (e.g. "x-api-key"), the raw key is sent on that header instead.

Override authenticate/2 only for unusual schemes like request signing or token refresh:

# Custom authentication
@impl true
def authenticate(req, opts) do
  with {:ok, key} <- Omni.Provider.resolve_auth(opts.api_key) do
    {:ok, Req.Request.put_header(req, "x-custom-auth", sign(key))}
  end
end

API keys are resolved in priority order:

1. Call-site opts — :api_key passed directly to generate_text/3 or stream_text/3
2. Application config — config :omni, MyProvider, api_key: ...
3. Provider default — the :api_key value from config/0

All three tiers accept the same value types — see resolve_auth/1.

Config keys

config/0 returns a map with the following keys:

• :base_url (required) — the service's base URL
• :api_key — default API key (see resolve_auth/1 for accepted formats)
• :auth_header — custom header name for the API key; when set, the raw key is sent on this header instead of as a Bearer token on "authorization"
• :headers — additional headers to include on every request (map)

These values serve as defaults. Users can override :base_url, :api_key, and :headers at the application config level or at the call site.

Multi-dialect providers

Some providers act as gateways to multiple upstream APIs, each with its own wire format. For example, OpenCode Zen routes Claude models through the Anthropic Messages format and GPT models through OpenAI Responses.

These providers omit the :dialect option:

defmodule MyApp.Providers.Gateway do
  use Omni.Provider, id: :gateway

  @impl true
  def config do
    %{base_url: "https://gateway.example.com", api_key: {:system, "GW_KEY"}}
  end
end

The model source resolves each model's dialect from its catalog data (see Omni.Sources.ModelsDev for how the default source infers dialects). The provider's declared dialect is only a fallback for models without one — when dialect/0 returns nil, a model whose dialect the source cannot resolve is skipped with a warning.

Model sources

Model catalog data comes from a pluggable source — see Omni.Source. load_models/2 resolves the source per call, in priority order:

1. Per-module config — config :omni, Omni.Providers.OpenAI, source: ...
2. Call-site opt — load_models(__MODULE__, source: ...) (a provider author's default for that provider)
3. Global config — config :omni, :models, source: ...
4. Default — Omni.Sources.ModelsDev (bundled models.dev snapshot)

A source value is a module or a {module, opts} tuple. Unlike API keys, the user's provider-specific config beats the call site here, because the call site is provider-author code rather than end-user code.

Sources look the provider up by its canonical id — the module's id/0 by default. To load from a different catalog entry, set provider_id: in the source's opts:

config :omni, MyApp.Providers.Gateway,
  source: {Omni.Sources.ModelsDev, provider_id: :mistral}

Choosing a dialect

Pick the dialect that matches your provider's wire format:

• Omni.Dialects.OpenAICompletions — OpenAI Chat Completions format, used by the majority of providers (Groq, Together, Fireworks, DeepSeek, etc.)
• Omni.Dialects.OpenAIResponses — OpenAI's newer Responses API format
• Omni.Dialects.AnthropicMessages — Anthropic Messages format
• Omni.Dialects.GoogleGemini — Google Gemini format
• Omni.Dialects.OllamaChat — Ollama native chat format (NDJSON streaming)

If your provider speaks a format not listed here, you'll need to implement a new dialect — see Omni.Dialect for the behaviour specification.

Registering a provider

All built-in providers are loaded at startup. To restrict which providers load, or to add custom ones, set the providers: key of the :models config to a list of provider modules — :builtins (the default) names all built-ins and may also appear inside the list:

config :omni, :models,
  providers: [
    Omni.Providers.Anthropic,
    Omni.Providers.OpenAI,
    MyApp.Providers.Acme
  ]

# or: all built-ins plus a custom provider
config :omni, :models,
  providers: [:builtins, MyApp.Providers.Acme]

To load a provider at runtime without restarting:

Omni.Provider.load([MyApp.Providers.Acme])

The provider's models are then available under its id, e.g. Omni.get_model(:acme, "acme-7b").