Configuration

LLM provides multiple ways to configure API keys, HTTP clients, and runtime options.

API keys

Application config

The recommended approach for production. Set keys in your config files:

# config/config.exs
config :llm, :providers,
  openai: [api_key: "sk-..."],
  anthropic: [api_key: "sk-ant-..."],
  gemini: [api_key: "AIza..."]

For runtime configuration (e.g., from environment variables):

# config/runtime.exs
config :llm, :providers,
  openai: [api_key: System.fetch_env!("OPENAI_API_KEY")],
  anthropic: [api_key: System.fetch_env!("ANTHROPIC_API_KEY")]

Runtime keys

Store keys at runtime using LLM.put_key/2. Keys are stored in the process dictionary:

LLM.put_key(:openai, "sk-...")
LLM.put_key(:anthropic, "sk-ant-...")

Per-request keys

Pass an API key directly in the provider tuple:

LLM.generate("Hello",
  provider: {LLM.Provider.OpenAI, api_key: "sk-..."},
  model: "gpt-4"
)

Key resolution order

When resolving an API key for a provider:

1. Per-request key (in {module, opts} tuple)
2. Process dictionary (LLM.put_key/2)
3. Application config (config :llm, :providers)

# This key takes precedence
LLM.generate("Hello",
  provider: {LLM.Provider.OpenAI, api_key: "per-request-key"},
  model: "gpt-4"
)

HTTP client

The default HTTP client is LLM.HTTPClient.Req, which wraps the Req library.

Custom HTTP client

Implement the LLM.HTTPClient behaviour:

defmodule MyApp.HTTPClient do
  @behaviour LLM.HTTPClient

  @impl true
  def request(req) do
    # Add custom headers, logging, retries, etc.
    req
    |> Req.Request.put_header("x-custom-header", "value")
    |> Req.request()
  end
end

Configure it:

# config/config.exs
config :llm, :http_client, MyApp.HTTPClient

Testing with Mox

For tests, swap in a mock client:

# test/test_helper.exs
Mox.defmock(LLM.HTTPClient.Mock, for: LLM.HTTPClient)

# config/test.exs
config :llm, :http_client, LLM.HTTPClient.Mock

Then set up expectations in your tests:

import Mox

test "generates text" do
  expect(LLM.HTTPClient.Mock, :request, fn req ->
    # Return a mock response
    {:ok, %Req.Response{status: 200, body: %{"choices" => [...]}}}
  end)

  {:ok, response} = LLM.generate("Hello", provider: :openai, model: "gpt-4")
end

Provider options

Options passed to generate/2 or stream/2:

Custom base URL

Use the config map provider to point to custom endpoints:

# Local Ollama
LLM.generate("Hello",
  provider: %{adapter: LLM.Adapter.OpenAI, base_url: "http://localhost:11434/v1"},
  model: "llama3"
)

# Custom proxy
LLM.generate("Hello",
  provider: %{
    adapter: LLM.Adapter.OpenAI,
    base_url: "https://my-proxy.example.com/v1",
    api_key: "my-key"
  },
  model: "gpt-4"
)

Model listing

Some providers support listing available models:

{:ok, models} = LLM.models(provider: :openai)

Enum.each(models, fn model ->
  IO.puts("#{model.id}: #{model.description}")
end)

Model info structure:

%{
  id: "gpt-4",
  name: "GPT-4",
  description: "Large multimodal model",
  context_window: 128_000,
  max_output: 4_096,
  capabilities: [:text, :vision, :tools]
}

Environment variables

A common pattern is to read API keys from environment variables:

# config/runtime.exs
config :llm, :providers,
  openai: [api_key: System.fetch_env!("OPENAI_API_KEY")],
  anthropic: [api_key: System.fetch_env!("ANTHROPIC_API_KEY")],
  gemini: [api_key: System.fetch_env!("GEMINI_API_KEY")]

Or use LLM.put_key/2 at application startup:

# lib/my_app/application.ex
def start(_type, _args) do
  LLM.put_key(:openai, System.fetch_env("OPENAI_API_KEY"))
  LLM.put_key(:anthropic, System.fetch_env("ANTHROPIC_API_KEY"))
  # ...
end

Telemetry

LLM optionally depends on :telemetry for instrumentation. If enabled, the following events are emitted:

To enable telemetry:

{:telemetry, "~> 1.0"}

See the Telemetry docs for how to attach handlers.