@spec generate(String.t() | LLM.Context.t(), [generate_option()], keyword()) ::
  {:ok, LLM.Response.t()} | {:error, term()}

Generate text (non-streaming). Returns the final response.

Sends a regular HTTP request and decodes the provider's full response. Executes tool calls with follow-up regular HTTP requests when present.

Callbacks can be placed in opts (2nd arg) or passed separately as a third keyword argument (which takes precedence):

# on_message in opts
{:ok, response} = LLM.generate("What is Elixir?",
  provider: :openai,
  model: "gpt-4",
  on_message: fn msg -> IO.inspect(msg.role) end
)

# on_message as separate third argument
{:ok, response} = LLM.generate("What is Elixir?",
  [provider: :openai, model: "gpt-4"],
  on_message: fn msg -> IO.inspect(msg.role) end
)

Callbacks

• :on_message — called once per completed LLM.Message, fn message -> ... end

When schema: is set, the model is asked to return JSON matching the given schema. On success, response.parsed contains the decoded map and the last assistant message in response.messages also carries :parsed. Tool auto-execution is disabled for that request since the response is structured data, not a tool call round-trip.

Structured output

{:ok, response} = LLM.generate("Extract the name and age.",
  provider: :openai,
  model: "gpt-4o",
  schema: %{
    "name" => "person",
    "schema" => %{
      "type" => "object",
      "properties" => %{
        "name" => %{"type" => "string"},
        "age" => %{"type" => "integer"}
      },
      "required" => ["name", "age"]
    }
  }
)
response.parsed  #=> %{"name" => "Alice", "age" => 30}

Pass a bare JSON Schema map to use "output" as the default name:

schema: %{"type" => "object", "properties" => %{...}}

The schema is passed through to the provider unchanged — it must be valid for the target provider's structured-output feature (schema requirements vary by provider).

@spec generate!(String.t() | LLM.Context.t(), [generate_option()], keyword()) ::
  LLM.Response.t()

Generate text, raising on error.

@spec get_key(atom()) :: String.t() | nil

Get an API key, checking process dictionary first, then application config.

@spec models(keyword()) :: {:ok, [LLM.Adapter.model_info()]} | {:error, term()}

List available models from a provider.

Returns {:ok, models} on success, where models is a list of model info maps. Returns {:error, reason} on failure.

Options

• :provider - provider preset atom, module, or config map (defaults to :openai)

Examples

{:ok, models} = LLM.models(provider: :openai)
{:ok, models} = LLM.models(provider: :anthropic)

@spec providers() :: [atom()]

List available provider presets.

@spec put_key(atom(), String.t()) :: :ok

Store an API key at runtime.

@spec stream(String.t() | LLM.Context.t(), keyword(), keyword()) ::
  {:ok, LLM.Response.t()} | {:error, term()}

Stream a prompt and return the final response.

Returns {:ok, response} on success, {:error, reason} on failure.

Callbacks can be placed in opts (2nd arg) or passed separately as a third keyword argument (which takes precedence):

# Callbacks in opts
{:ok, response} = LLM.stream("Tell me a story",
  provider: :openai,
  model: "gpt-4",
  on_chunk: &IO.write/1
)

# Callbacks as separate third argument
{:ok, response} = LLM.stream("Tell me a story",
  [provider: :openai, model: "gpt-4"],
  on_chunk: &IO.write/1
)

Callbacks

• :on_chunk — called for each chunk, fn chunk -> ... end
• :on_message — called once per completed LLM.Message, fn message -> ... end

For manual stream control use LLM.Stream.start/2 and LLM.Stream.collect/2 directly.

@spec stream!(String.t() | LLM.Context.t(), keyword(), keyword()) :: LLM.Response.t()

Stream a prompt and return the response, raising on error.