LLMProxy.Providers.Behaviour behaviour (llm_proxy v0.1.1)

Copy Markdown View Source

Behaviour for LLM provider implementations.

Providers receive native request bodies and return LLMProxy.Providers.Result structs. Wire JSON stays at protocol/provider boundaries.

Summary

Types

call_result()
stream()
stream_result()
usage()

Callbacks

call(body, user_id)

Non-streaming call. Body is in the provider's native protocol format.

call_native(body, user_id)

Non-streaming native passthrough. Routes pass the validated native wire body explicitly.

extract_usage(response)

Extract usage from a non-streaming response body

models()

List of model IDs this provider handles

name()

Provider name, e.g. "anthropic", "openrouter"

native_protocol()

The native API protocol this provider speaks (:openai or :anthropic)

stream(body, user_id)

Streaming call. Body is in the provider's native protocol format.

stream_error(reason, token)

Project an exception raised while lazily consuming a provider stream

stream_native(body, user_id)

Streaming native passthrough. Routes pass the validated native wire body explicitly.

to_openai_response(response, model)

Convert native response to OpenAI chat completion format

Types

call_result()

@type call_result() ::
  {:ok, LLMProxy.Providers.Result.t()} | {:error, LLMProxy.Providers.Result.t()}

stream()

@type stream() :: Enumerable.t()

stream_result()

@type stream_result() ::
  {:ok, LLMProxy.Providers.Result.t()} | {:error, LLMProxy.Providers.Result.t()}

usage()

@type usage() :: LLMProxy.Usage.t()

Callbacks

call(body, user_id)

@callback call(body :: map(), user_id :: String.t()) :: call_result()

Non-streaming call. Body is in the provider's native protocol format.

call_native(body, user_id)

(optional) 
@callback call_native(body :: map(), user_id :: String.t()) :: call_result()

Non-streaming native passthrough. Routes pass the validated native wire body explicitly.

extract_usage(response)

@callback extract_usage(response :: map()) :: usage()

Extract usage from a non-streaming response body

models()

@callback models() :: [String.t()]

List of model IDs this provider handles

name()

@callback name() :: String.t()

Provider name, e.g. "anthropic", "openrouter"

native_protocol()

@callback native_protocol() :: :openai | :anthropic

The native API protocol this provider speaks (:openai or :anthropic)

stream(body, user_id)

@callback stream(body :: map(), user_id :: String.t()) :: stream_result()

Streaming call. Body is in the provider's native protocol format.

stream_error(reason, token)

(optional) 
@callback stream_error(reason :: term(), token :: map() | nil) ::
  LLMProxy.Providers.Result.t()

Project an exception raised while lazily consuming a provider stream

stream_native(body, user_id)

(optional) 
@callback stream_native(body :: map(), user_id :: String.t()) :: stream_result()

Streaming native passthrough. Routes pass the validated native wire body explicitly.

to_openai_response(response, model)

@callback to_openai_response(response :: map(), model :: String.t()) :: map()

Convert native response to OpenAI chat completion format