LLM (LLM v0.1.3)

Lightweight Elixir client for LLM APIs.

Supports 4 adapters covering 99% of providers:

OpenAI Chat Completions — /v1/chat/completions (also works with Ollama, Groq, OpenRouter, DeepSeek, xAI, etc.)
OpenAI Responses — /v1/responses
Anthropic Messages — /v1/messages
Gemini — /v1beta/models/{model}:generateContent

Quick Start

# Simple text generation
{:ok, response} = LLM.generate("What is Elixir?",
  provider: :openai,
  model: "gpt-4"
)
response.message.content  #=> "Elixir is a..."

# Streaming
{:ok, stream} = LLM.stream("Tell me a story",
  provider: :anthropic,
  model: "claude-sonnet-4-5-20250514"
)
{:ok, response} = LLM.Stream.collect(stream, on_chunk: &IO.write/1)

# Using provider modules
{:ok, response} = LLM.generate("Hello",
  provider: LLM.Provider.OpenAI,
  model: "gpt-4"
)

# With explicit API key
{:ok, response} = LLM.generate("Hello",
  provider: {LLM.Provider.OpenAI, api_key: "sk-..."},
  model: "gpt-4"
)

# Custom provider (runtime)
{:ok, response} = LLM.generate("Hello",
  provider: %{
    adapter: LLM.Adapter.Anthropic,
    base_url: "https://my-proxy.com",
    api_key: "sk-ant-..."
  },
  model: "claude-sonnet-4-5-20250514"
)

# With tools
{:ok, response} = LLM.generate("Read mix.exs",
  provider: :openai,
  model: "gpt-4",
  tools: [MyApp.ReadFileTool]
)

Configuration

# config/config.exs
config :llm, :providers,
  openai: [api_key: "sk-..."],
  anthropic: [api_key: "sk-ant-..."]

# Or at runtime
LLM.put_key(:openai, "sk-...")

Provider

A provider can be:

An atom preset (:openai, :anthropic, :gemini, :openrouter, :openai_responses)
A provider module (LLM.Provider.OpenAI, LLM.Provider.Anthropic, etc.)
A tuple {module, opts} for runtime API key ({LLM.Provider.OpenAI, api_key: "sk-..."})
A map with :adapter, :base_url, and optionally :api_key

Summary

Functions

generate(prompt, opts \\ [])

Generate text (non-streaming). Returns the final response.

generate!(prompt, opts \\ [])

Generate text, raising on error.

get_key(provider_name)

Get an API key, checking process dictionary first, then application config.

models(opts \\ [])

List available models from a provider.

providers()

List available provider presets.

put_key(provider_name, api_key)

Store an API key at runtime.

stream(prompt, opts \\ [])

Stream a prompt, returning a stream handle.

Types

stream_option()

@type stream_option() ::
  {:provider, atom() | map() | module()}
  | {:model, String.t()}
  | {:max_tokens, non_neg_integer()}
  | {:temperature, float()}
  | {:thinking, atom() | map()}
  | {:tools, [module() | LLM.Tool.t() | {module(), map()}]}
  | {:auto_tools, boolean()}
  | {:max_rounds, non_neg_integer()}
  | {:on_chunk, (LLM.Stream.chunk_type() -> any())}
  | {:on_message, (LLM.Message.t() -> any())}
  | {:system, String.t()}
  | {:messages, [LLM.Message.t()]}
  | {:structured_output, map()}

Functions

generate(prompt, opts \\ [])

@spec generate(String.t() | LLM.Context.t(), [stream_option()]) ::
  {:ok, LLM.Response.t()} | {:error, term()}

Generate text (non-streaming). Returns the final response.

Automatically collects the stream and executes tool calls if present.

When structured_output: is set, the model is asked to return JSON matching the given schema. On success, response.parsed contains the decoded map. Tool auto- execution is disabled for that request since the response is structured data, not a tool call round-trip.

Structured output

{:ok, response} = LLM.generate("Extract the name and age.",
  provider: :openai,
  model: "gpt-4o",
  structured_output: %{
    "name" => "person",
    "schema" => %{
      "type" => "object",
      "properties" => %{
        "name" => %{"type" => "string"},
        "age" => %{"type" => "integer"}
      },
      "required" => ["name", "age"]
    }
  }
)
response.parsed  #=> %{"name" => "Alice", "age" => 30}

Pass a bare JSON Schema map to use "output" as the default name:

structured_output: %{"type" => "object", "properties" => %{...}}

The schema is passed through to the provider unchanged — it must be valid for the target provider's structured-output feature (schema requirements vary by provider).