LLMProxy.Provider (llm_proxy v0.1.0)

Core in-process LLMProxy provider and ReqLLM adapter.

HTTP routes and local Elixir calls use this module as the execution boundary. The ReqLLM callbacks expose the same execution path through ReqLLM's provider API.

Summary

Types

call_error()

Functions

attach(request, model_input, user_opts)

Default implementation of attach/3.

attach_stream(model, context, opts, finch_name)

Default implementation of attach_stream/4.

base_url()

build_body(request)

Default implementation of build_body/1.

call(request, actor_or_key, opts \\ [])

call_native(request, actor_or_key, opts \\ [])

chat(messages, opts \\ [])

chat_request(messages, opts)

decode_response(request_response)

Default implementation of decode_response/1.

decode_stream_event(event, model)

Default implementation of decode_stream_event/2.

default_base_url()

encode_body(request)

Default implementation of encode_body/1.

extract_usage(body, model)

Default implementation of extract_usage/2.

prepare_request(operation, model_spec, input, opts)

Default implementation of prepare_request/4.

provider_extended_generation_schema()

provider_id()

provider_schema()

stream(request, actor_or_key, opts \\ [])

stream_native(request, actor_or_key, opts \\ [])

supported_provider_options()

translate_options(operation, model, opts)

Default implementation of translate_options/3.

Types

call_error()

@type call_error() ::
  {:request, LLMProxy.Protocol.Request.Error.t()}
  | {:permission, String.t()}
  | {:not_found, String.t()}
  | {:missing_api_key, term()}
  | {:invalid_api_key, String.t()}
  | {:guardrail, term()}
  | {:provider, LLMProxy.Providers.Result.t()}

Functions

attach(request, model_input, user_opts)

Default implementation of attach/3.

Sets up Bearer token authentication and standard pipeline steps.

attach_stream(model, context, opts, finch_name)

Default implementation of attach_stream/4.

Builds complete streaming requests using OpenAI-compatible format.

base_url()

build_body(request)

Default implementation of build_body/1.

Builds request body using OpenAI-compatible format for chat and embedding operations.

call(request, actor_or_key, opts \\ [])

@spec call(
  LLMProxy.Protocol.Request.t(),
  LLMProxy.Actor.t() | map() | String.t(),
  keyword()
) ::
  {:ok, LLMProxy.Response.t()} | {:error, call_error()}

call_native(request, actor_or_key, opts \\ [])

@spec call_native(
  LLMProxy.Protocol.Request.t(),
  LLMProxy.Actor.t() | map() | String.t(),
  keyword()
) ::
  {:ok, LLMProxy.Providers.Result.t()} | {:error, call_error()}

chat(messages, opts \\ [])

@spec chat(
  String.t() | list() | ReqLLM.Context.t(),
  keyword()
) ::
  {:ok, LLMProxy.Response.t()}
  | {:error, call_error() | {:request, LLMProxy.Protocol.Request.Error.t()}}

chat_request(messages, opts)

@spec chat_request(
  String.t() | list() | ReqLLM.Context.t(),
  keyword()
) ::
  {:ok, LLMProxy.Protocol.Request.t()}
  | {:error, LLMProxy.Protocol.Request.Error.t() | term()}

decode_response(request_response)

Default implementation of decode_response/1.

Handles success/error responses with standard ReqLLM.Response creation.

decode_stream_event(event, model)

Default implementation of decode_stream_event/2.

Decodes SSE events using OpenAI-compatible format.

default_base_url()

encode_body(request)

Default implementation of encode_body/1.

Encodes request body using OpenAI-compatible format for chat and embedding operations.

extract_usage(body, model)

Default implementation of extract_usage/2.

Extracts usage data from standard usage field in response body.

prepare_request(operation, model_spec, input, opts)

Default implementation of prepare_request/4.

Handles :chat, :object, and :embedding operations using OpenAI-compatible patterns.

provider_extended_generation_schema()

provider_id()

provider_schema()

stream(request, actor_or_key, opts \\ [])

@spec stream(
  LLMProxy.Protocol.Request.t(),
  LLMProxy.Actor.t() | map() | String.t(),
  keyword()
) ::
  {:ok, LLMProxy.Providers.Result.t()} | {:error, call_error()}

stream_native(request, actor_or_key, opts \\ [])

@spec stream_native(
  LLMProxy.Protocol.Request.t(),
  LLMProxy.Actor.t() | map() | String.t(),
  keyword()
) ::
  {:ok, LLMProxy.Providers.Result.t()} | {:error, call_error()}

supported_provider_options()

translate_options(operation, model, opts)

Default implementation of translate_options/3.

Pass-through implementation that returns options unchanged.