Nous.Provider behaviour (nous v0.16.4)

View Source

Behaviour for LLM provider implementations.

Provides a declarative way to define providers with common functionality injected via the use macro.

Usage

defmodule Nous.Providers.OpenAI do
  use Nous.Provider,
    id: :openai,
    default_base_url: "https://api.openai.com/v1",
    default_env_key: "OPENAI_API_KEY"

  @impl true
  def chat(params, opts \\ []) do
    # Implementation
  end

  @impl true
  def chat_stream(params, opts \\ []) do
    # Implementation
  end
end

Callbacks

Providers must implement:

  • chat/2 - Non-streaming chat completion (low-level HTTP)
  • chat_stream/2 - Streaming chat completion (low-level HTTP)

Optional (have default implementations):

  • request/3 - High-level request with message conversion, telemetry, error wrapping
  • request_stream/3 - High-level streaming with message conversion, telemetry
  • count_tokens/1 - Token counting (defaults to rough estimate)

Injected Functions

The use macro injects:

  • provider_id/0 - Returns the provider atom ID
  • default_base_url/0 - Returns the default API base URL
  • default_env_key/0 - Returns the environment variable name for API key
  • api_key/1 - Returns the API key from opts, env, or config
  • base_url/1 - Returns base URL from opts, config, or default
  • request/3 - High-level request (can be overridden)
  • request_stream/3 - High-level streaming request (can be overridden)

Summary

Callbacks

chat(params, opts)

Make a non-streaming chat request (low-level HTTP).

chat_stream(params, opts)

Make a streaming chat request (low-level HTTP).

count_tokens(messages)

Count tokens in messages (can be an estimate).

default_base_url()

Returns the default API base URL

default_env_key()

Returns the environment variable name for the API key

provider_id()

Returns the provider identifier atom

request(model, messages, settings)

High-level request with message conversion, telemetry, and error wrapping.

request_stream(model, messages, settings)

High-level streaming request with message conversion and telemetry.

Callbacks

chat(params, opts)

@callback chat(params :: map(), opts :: keyword()) :: {:ok, map()} | {:error, term()}

Make a non-streaming chat request (low-level HTTP).

Parameters

  • params - Request parameters (model, messages, etc.)
  • opts - Options including :api_key, :base_url, :timeout

Returns

  • {:ok, response} - Parsed response body
  • {:error, reason} - Error with reason

chat_stream(params, opts)

@callback chat_stream(params :: map(), opts :: keyword()) ::
  {:ok, Enumerable.t()} | {:error, term()}

Make a streaming chat request (low-level HTTP).

Parameters

  • params - Request parameters (model, messages, etc.)
  • opts - Options including :api_key, :base_url, :timeout

Returns

  • {:ok, stream} - Enumerable of parsed events
  • {:error, reason} - Error with reason

count_tokens(messages)

(optional) 
@callback count_tokens(messages :: list()) :: integer()

Count tokens in messages (can be an estimate).

Optional callback - defaults to rough estimation if not implemented.

default_base_url()

@callback default_base_url() :: String.t()

Returns the default API base URL

default_env_key()

@callback default_env_key() :: String.t()

Returns the environment variable name for the API key

provider_id()

@callback provider_id() :: atom()

Returns the provider identifier atom

request(model, messages, settings)

(optional) 
@callback request(model :: Nous.Model.t(), messages :: list(), settings :: map()) ::
  {:ok, Nous.Message.t()} | {:error, term()}

High-level request with message conversion, telemetry, and error wrapping.

Parameters

  • model - Model configuration struct
  • messages - List of messages in internal format
  • settings - Request settings (temperature, max_tokens, tools, etc.)

Returns

  • {:ok, message} - Parsed response as Message struct
  • {:error, error} - Wrapped error

request_stream(model, messages, settings)

(optional) 
@callback request_stream(model :: Nous.Model.t(), messages :: list(), settings :: map()) ::
  {:ok, Enumerable.t()} | {:error, term()}

High-level streaming request with message conversion and telemetry.

Parameters

  • model - Model configuration struct
  • messages - List of messages in internal format
  • settings - Request settings

Returns

  • {:ok, stream} - Stream of normalized events
  • {:error, error} - Wrapped error