Modules
Elixir-native LLM gateway for in-process, ReqLLM, SafeRPC, and HTTP calls.
Usage accounting workflow for key counters, usage rows, optional traces, and user-message logging.
Authenticated caller identity used below transport boundaries.
Service-owned Incant admin interface for LLMProxy.
Incant action callbacks for connecting OpenAI Codex OAuth accounts.
Input required to complete an OpenAI Codex OAuth login.
OpenAI Codex OAuth authorization details returned to the operator.
Operations dashboard for LLMProxy usage, spend, and token activity.
Incant authorization policy for the LLMProxy admin surface.
Admin resource for LLMProxy API keys.
Admin resource for logged user messages.
Admin resource for upstream provider tokens.
Admin resource for recorded LLM traces.
OTP application callback that starts LLMProxy storage, routing state, token pools, RPC, and HTTP serving.
Behaviour for deterministic response cache adapters.
Resolves configured and per-request cache policy into a strict runtime policy struct.
Runtime dispatcher for optional cache adapters, cache-key generation, and cache writes.
Public model catalog for aliases and deployment metadata.
Internal catalog deployment target for a public model alias.
Internal catalog entry for a public model alias.
Runtime accessors and normalization helpers for LLMProxy application configuration.
Boundary parser for catalog application configuration.
Release config provider for optional standalone LLMProxy TOML config.
TOML decoder for standalone LLMProxy configuration files.
Runtime drain control for deployment-safe LLMProxy shutdowns.
Behaviour for host-defined request/response policy hooks.
Runs configured LLMProxy.Guardrail callbacks around requests, responses, and stream events.
Req client factory that applies LLMProxy's HTTP instrumentation and test plug configuration.
Route table shared by standalone Plug routing and Phoenix router embedding macros.
Standalone Plug router for LLMProxy health checks, core API routes, setup routes, and dynamic extensions.
OpenAI Chat Completions route adapter for authenticated HTTP requests and SSE streams.
Registry for dynamically registered routes from optional packages.
Trace feedback endpoint for attaching user ratings and comments to request IDs or trace IDs.
Anthropic Messages route adapter that preserves provider-native request and response shapes.
Serves the OpenAI-compatible model list endpoint.
Serves OpenAI-compatible moderation requests.
Parsed moderation request body.
Shared provider-passthrough route flow.
Route callbacks used to render and classify passthrough-route errors.
Route callbacks used to render passthrough non-streaming and streaming results.
OpenAI Responses route adapter that passes native request and stream events through LLMProxy execution.
Optional setup-helper routes for install scripts, model lists, and client configuration snippets.
Parsed setup endpoint authentication.
Usage limit for API keys.
LLMDB integration for provider model lists and pricing lookup used by routing and accounting.
Local operational RPC surface for deployment lifecycle control.
Phoenix router macros that mount LLMProxy route groups inside host applications.
Extracts and validates API key from Authorization header or x-api-key.
Assigns :api_key to the conn.
Rejects new user work while LLMProxy is draining.
Parses authenticated JSON request bodies with a bounded limit and a structured oversized response.
Checks token/message/cache quotas for the authenticated API key. Must run after LLMProxy.Plugs.Auth.
Model pricing lookup and cost calculation.
Per-million-token pricing rates used by LLMProxy cost accounting.
Protocol detection and conversion between LLM API formats.
Anthropic Messages API protocol.
OpenAI chat completion protocol.
Boundary parser that normalizes OpenAI Chat, Anthropic Messages, and OpenAI Responses requests.
Structured request-parse error returned by protocol request boundary parsers.
Core in-process LLMProxy provider and ReqLLM adapter.
Built-in Anthropic Messages provider for native and OpenAI-compatible LLMProxy execution.
Normalized provider deployment attempt used by routing, fallback, cache keys, and circuit breakers.
Behaviour for LLM provider implementations.
GenServer-backed deployment circuit breaker for routing skips, half-open probes, and cooldowns.
Executes provider attempts with timeout, retry, fallback, telemetry, and circuit-breaker handling.
Converts upstream HTTP responses and exceptions into LLMProxy.Providers.Result values.
Built-in OpenAI provider backed by the shared OpenAI-compatible provider implementation.
Adapter for the ChatGPT Codex backend exposed by ReqLLM.
Converts ReqLLM Codex stream chunks into LLMProxy stream events.
OAuth credential refresh boundary for the OpenAI Codex provider.
Interactive OpenAI Codex OAuth login for standalone LLMProxy releases.
Codex access-token JWT claims used by LLMProxy.
OpenAI auth claim payload embedded in the Codex access-token JWT.
OpenAI OAuth token endpoint response.
ReqLLM OpenAI Codex OAuth refresh response.
Shared HTTP client implementation for upstream providers with OpenAI-compatible chat endpoints.
Macro frontend for defining OpenAI-compatible upstream providers.
Built-in OpenRouter provider backed by the shared OpenAI-compatible provider implementation.
Provider discovery via :persistent_term.
Configuration-driven upstream provider backed by ReqLLM.
Provider execution result tagged by response kind.
GenServer-backed routing strategy that rotates deployments within each order group by model name.
SafeRPC socket server for LLMProxy local RPC surfaces.
Release-safe operational tasks for standalone LLMProxy deployments.
Internal non-stream LLMProxy response.
Public router facade for embedding LLMProxy.
Ecto schema for LLMProxy API keys, quotas, budgets, model access, and aggregate usage counters.
Ecto schema for captured user-message snippets used by operations and admin views.
Ecto schema for upstream provider API keys, OAuth tokens, labels, proxies, and enabled state.
Ecto schema for non-provider service usage counters tied to an API key and endpoint.
Ecto schema for optional traced request and response payloads with usage and latency metadata.
Ecto schema for user feedback associated with traces or request IDs.
Ecto schema for per-request token, cost, latency, provider, tag, and metadata usage records.
Public persistence facade for keys, usage, quotas, tokens, traces, and feedback.
Behaviour for persistence adapters that back LLMProxy storage operations.
Context functions for all database operations: keys, usage, quotas, tokens.
Ecto type for storing map and list values as JSON strings across supported SQL adapters.
Starts the bundled QuackDB server and wires its generated token into the bundled Ecto repo.
Facade for the configured LLMProxy Ecto repo.
Bundled QuackDB Ecto repo for production DuckDB-backed storage.
Bundled SQLite Ecto repo for local development, tests, and standalone storage.
SQL adapter detection and support checks for storage queries and migrations.
Server-sent stream event with protocol wire-event constructors.
Adds periodic heartbeat markers while an upstream enumerable is silent.
Helpers for writing SSE events to a Plug.Conn chunked response.
Telemetry and OpenTelemetry helpers for provider routing and execution spans.
Manages provider tokens with rate-limit cooldowns.
Runtime state for provider-token cooldown timestamps.
Request and trace ID generation plus Plug response-header propagation.
Token usage accounting and protocol usage-map rendering.
Mix Tasks
Installs LLMProxy embedder migration aliases into an existing Mix project.