Getting Started

Copy Markdown View Source

Installation

Add :llm to your list of dependencies in mix.exs:

def deps do
  [
    {:llm, "~> 0.1.0"}
  ]
end

The library uses Req under the hood for HTTP requests. It is included as a dependency automatically.

Configuration

# config/config.exs
config :llm, :providers,
  openai: [api_key: "sk-..."],
  anthropic: [api_key: "sk-ant-..."]

Runtime config (for development or per-process keys)

LLM.put_key(:openai, "sk-...")
LLM.put_key(:anthropic, "sk-ant-...")

Per-request API key

LLM.generate("Hello",
  provider: {LLM.Provider.OpenAI, api_key: "sk-..."},
  model: "gpt-4"
)

See the Configuration guide for all options.

Your first request

# Simple text generation
{:ok, response} = LLM.generate("What is Elixir?",
  provider: :openai,
  model: "gpt-4"
)

response.message.content
#=> "Elixir is a functional, concurrent, general-purpose programming language..."

Streaming

{:ok, stream} = LLM.stream("Tell me a short joke",
  provider: :openai,
  model: "gpt-4"
)

# Collect with a callback
{:ok, response} = LLM.Stream.collect(stream, on_chunk: fn chunk ->
  IO.write(chunk.text)
end)

# Or collect silently
{:ok, response} = LLM.Stream.collect(stream)
response.message.content
#=> "Why do Elixir developers... "

Using provider modules

Instead of atoms, you can pass provider modules directly:

{:ok, response} = LLM.generate("Hello",
  provider: LLM.Provider.OpenAI,
  model: "gpt-4"
)

{:ok, response} = LLM.generate("Hello",
  provider: LLM.Provider.Anthropic,
  model: "claude-sonnet-4-5-20250514"
)

Using tools

Define a tool module and pass it to generate/2:

defmodule MyApp.MathTool do
  @behaviour LLM.Tool

  @impl true
  def name, do: "add"

  @impl true
  def description, do: "Add two numbers"

  @impl true
  def input_schema do
    %{
      "type" => "object",
      "properties" => %{
        "a" => %{"type" => "number"},
        "b" => %{"type" => "number"}
      },
      "required" => ["a", "b"]
    }
  end

  @impl true
  def execute(%{"a" => a, "b" => b}, _context) do
    {:ok, to_string(a + b)}
  end
end

{:ok, response} = LLM.generate("What is 2 + 3?",
  provider: :openai,
  model: "gpt-4",
  tools: [MyApp.MathTool]
)

response.message.content
#=> "The result of 2 + 3 is 5."

Tool calls are executed automatically during generate/2. See the Tools guide for more details.

Structured output

Pass a JSON Schema to structured_output: and the model returns JSON matching that shape. The decoded map is in response.parsed:

schema = %{
  "type" => "object",
  "properties" => %{
    "name"      => %{"type" => "string"},
    "sentiment" => %{"type" => "string", "enum" => ["positive", "negative", "neutral"]}
  },
  "required" => ["name", "sentiment"]
}

{:ok, response} = LLM.generate("Analyse: 'Great product, love it!' — reviewer: Alice",
  provider: :openai,
  model: "gpt-4o",
  structured_output: schema
)

response.parsed
#=> %{"name" => "Alice", "sentiment" => "positive"}

See the Structured Output guide for schema formats, provider-specific notes, and error handling.

Error handling

All functions return {:ok, result} or {:error, reason} tuples. Use the bang versions for exceptions:

# With pattern matching
case LLM.generate("Hello", provider: :openai, model: "gpt-4") do
  {:ok, response} -> handle_response(response)
  {:error, reason} -> handle_error(reason)
end

# With bang version (raises on error)
response = LLM.generate!("Hello", provider: :openai, model: "gpt-4")

Listing providers and models

LLM.providers()
#=> [:openai, :openai_responses, :anthropic, :gemini, :openrouter]

{:ok, models} = LLM.models(provider: :openai)

Next steps

  • Providers — learn about all available providers and how to configure them
  • Streaming — work with streaming responses in detail
  • Tools — define tools with modules or inline functions
  • Structured Output — get typed JSON back from any provider
  • Configuration — API keys, HTTP client, and all options