Providers

LLM supports multiple providers through a unified interface. Each provider maps to an adapter that handles the wire format, SSE streaming, and authentication.

Built-in providers

Provider specification types

A provider can be specified in four ways:

1. Preset atom

The simplest option. Resolves to the built-in provider module:

LLM.generate("Hello", provider: :openai, model: "gpt-4")
LLM.generate("Hello", provider: :anthropic, model: "claude-sonnet-4-5-20250514")

2. Provider module

Pass the module directly. Useful when you want to be explicit:

LLM.generate("Hello", provider: LLM.Provider.OpenAI, model: "gpt-4")

3. Module with runtime options

A {module, opts} tuple lets you pass an API key (or other options) without polluting global config:

LLM.generate("Hello",
  provider: {LLM.Provider.OpenAI, api_key: "sk-..."},
  model: "gpt-4"
)

4. Config map

For fully custom providers (proxies, self-hosted endpoints, or providers not in the presets):

LLM.generate("Hello",
  provider: %{
    adapter: LLM.Adapter.OpenAI,
    base_url: "https://my-proxy.example.com/v1",
    api_key: "my-key"
  },
  model: "gpt-4"
)

The map must include :adapter and :base_url. The :api_key is optional.

Provider resolution

LLM.Provider.Resolver.resolve/1 converts any provider specification into a full config map:

config = LLM.Provider.Resolver.resolve(:openai)
#=> %{
#     adapter: LLM.Adapter.OpenAI,
#     base_url: "https://api.openai.com/v1",
#     api_key: "sk-...",
#     http_opts: []
#   }

Resolution order:

1. nil → raises ArgumentError
2. Map with :adapter and :base_url → used as-is
3. Map without required keys → raises ArgumentError
4. {module, opts} tuple → resolve module, merge opts
5. Atom → look up in LLM.Provider behaviour or built-in presets

Adding a custom provider

Implement the LLM.Provider behaviour:

defmodule MyApp.CustomProvider do
  @behaviour LLM.Provider

  @impl true
  def name, do: :custom

  @impl true
  def default_config do
    %{
      adapter: LLM.Adapter.OpenAI,  # reuse an existing adapter
      base_url: "https://my-api.example.com/v1",
      api_key: Application.get_env(:my_app, :custom_api_key)
    }
  end
end

Then use it:

LLM.generate("Hello", provider: MyApp.CustomProvider, model: "my-model")

OpenAI Chat Completions

Works with OpenAI, Ollama, Groq, DeepSeek, xAI, and any OpenAI-compatible API:

# Direct OpenAI
LLM.generate("Hello", provider: :openai, model: "gpt-4")

# Ollama (local)
LLM.generate("Hello",
  provider: %{adapter: LLM.Adapter.OpenAI, base_url: "http://localhost:11434/v1"},
  model: "llama3"
)

# Groq
LLM.generate("Hello",
  provider: %{adapter: LLM.Adapter.OpenAI, base_url: "https://api.groq.com/openai/v1", api_key: "gsk_..."},
  model: "llama-3.1-70b-versatile"
)

Anthropic Messages

LLM.generate("Hello", provider: :anthropic, model: "claude-sonnet-4-5-20250514")

Supports extended thinking:

LLM.generate("Explain quantum computing",
  provider: :anthropic,
  model: "claude-sonnet-4-5-20250514",
  thinking: :high
)

Gemini

LLM.generate("Hello", provider: :gemini, model: "gemini-2.0-flash")

OpenRouter

OpenRouter uses the OpenAI-compatible adapter:

LLM.generate("Hello", provider: :openrouter, model: "anthropic/claude-sonnet-4-5-20250514")

Thinking / reasoning

All providers that support reasoning accept the :thinking option:

# Discrete levels (provider-specific interpretation)
LLM.generate("Think step by step", provider: :openai, model: "o3-mini", thinking: :medium)
LLM.generate("Think step by step", provider: :anthropic, model: "claude-sonnet-4-5-20250514", thinking: :high)

# Custom budget (Anthropic)
LLM.generate("Think deeply",
  provider: :anthropic,
  model: "claude-sonnet-4-5-20250514",
  thinking: %{"type" => "enabled", "budget_tokens" => 50_000}
)

Thinking levels map to provider-specific configurations:

Next steps

• Streaming — work with streaming responses
• Tools — define and use tool calls
• Configuration — API keys and runtime options