LLM.Provider behaviour (LLM v0.1.3)

Copy Markdown View Source

Provider configuration behaviour.

A provider is either:

  • A map with :adapter, :base_url, and optionally :api_key
  • A module implementing the LLM.Provider behaviour

Built-in provider modules (LLM.Provider.OpenAI, LLM.Provider.Anthropic, etc.) implement this behaviour and provide sensible defaults for their respective APIs.

Implementing a custom provider

defmodule MyApp.Provider do
  @behaviour LLM.Provider

  @impl true
  def name, do: :my_custom

  @impl true
  def default_config do
    %{
      adapter: LLM.Adapter.OpenAI,
      base_url: "https://my-api.example.com/v1",
      api_key: Application.get_env(:my_app, :api_key)
    }
  end
end

Then use it:

LLM.generate("Hello", provider: MyApp.Provider, model: "my-model")

Summary

Types

config()
pricing()

Callbacks

default_config()

Return the default configuration map for this provider.

name()

Return the provider name as an atom (e.g., :openai, :anthropic).

pricing(model_id)

Return pricing per million tokens for a given model, or nil if unknown.

Types

config()

@type config() :: %{
  adapter: module(),
  base_url: String.t(),
  api_key: String.t() | nil,
  http_opts: keyword()
}

pricing()

@type pricing() :: %{
  input: number(),
  output: number(),
  cache_read: number(),
  cache_write: number()
}

Callbacks

default_config()

@callback default_config() :: config()

Return the default configuration map for this provider.

Must include :adapter and :base_url. The :api_key should be fetched from LLM.get_key/1 or application config.

name()

@callback name() :: atom()

Return the provider name as an atom (e.g., :openai, :anthropic).

pricing(model_id)

@callback pricing(model_id :: String.t()) :: pricing() | nil

Return pricing per million tokens for a given model, or nil if unknown.

Pricing is a map with cost per million tokens in USD:

%{
  input: 3.0,
  output: 15.0,
  cache_read: 0.3,
  cache_write: 3.75
}

This callback is optional. If not implemented, returns nil. Users can also pass :pricing in call-site options to override.