# llm_proxy v0.1.0 - Table of Contents

> Elixir-native LLM gateway for embedded and standalone deployments with provider routing, quotas, usage tracking, and OpenAI-compatible APIs.

## Pages

- [LLMProxy](readme.md)
- [Changelog](changelog.md)
- [Roadmap](roadmap.md)

- Introduction
  - [Getting Started](getting-started.md)
  - [Library Mode](library-mode.md)
  - [Standalone Mode](standalone-mode.md)

- Features
  - [Providers and Routing](providers-and-routing.md)
  - [Governance and Observability](governance-and-observability.md)
  - [Cache and Guardrails](cache-and-guardrails.md)
  - [Admin Integration](admin-integration.md)

- Deployment
  - [Standalone Deployment](standalone-deployment.md)

- Reference
  - [Configuration](configuration.md)
  - [HTTP API](http-api.md)

- Internals
  - [Architecture](architecture.md)

## Modules

- [LLMProxy.Accounting.UsageTracking](LLMProxy.Accounting.UsageTracking.md): Usage accounting workflow for key counters, usage rows, optional traces, and user-message logging.

- [LLMProxy.Admin](LLMProxy.Admin.md): Service-owned Incant admin interface for LLMProxy.
- [LLMProxy.Admin.CodexOAuth](LLMProxy.Admin.CodexOAuth.md): Incant action callbacks for connecting OpenAI Codex OAuth accounts.
- [LLMProxy.Admin.CodexOAuth.CompleteInput](LLMProxy.Admin.CodexOAuth.CompleteInput.md): Input required to complete an OpenAI Codex OAuth login.
- [LLMProxy.Admin.CodexOAuth.StartResult](LLMProxy.Admin.CodexOAuth.StartResult.md): OpenAI Codex OAuth authorization details returned to the operator.
- [LLMProxy.Admin.Dashboards.Operations](LLMProxy.Admin.Dashboards.Operations.md): Operations dashboard for LLMProxy usage, spend, and token activity.
- [LLMProxy.Admin.Policy](LLMProxy.Admin.Policy.md): Incant authorization policy for the LLMProxy admin surface.
- [LLMProxy.Admin.Resources.ApiKey](LLMProxy.Admin.Resources.ApiKey.md): Admin resource for LLMProxy API keys.
- [LLMProxy.Admin.Resources.Message](LLMProxy.Admin.Resources.Message.md): Admin resource for logged user messages.
- [LLMProxy.Admin.Resources.ProviderToken](LLMProxy.Admin.Resources.ProviderToken.md): Admin resource for upstream provider tokens.
- [LLMProxy.Admin.Resources.Trace](LLMProxy.Admin.Resources.Trace.md): Admin resource for recorded LLM traces.
- [LLMProxy.Application](LLMProxy.Application.md): OTP application callback that starts LLMProxy storage, routing state, token pools, RPC, and HTTP serving.

- [LLMProxy.Cache.Policy](LLMProxy.Cache.Policy.md): Resolves configured and per-request cache policy into a strict runtime policy struct.

- [LLMProxy.Cache.Runtime](LLMProxy.Cache.Runtime.md): Runtime dispatcher for optional cache adapters, cache-key generation, and cache writes.

- [LLMProxy.Config](LLMProxy.Config.md): Runtime accessors and normalization helpers for LLMProxy application configuration.

- [LLMProxy.Config.Catalog](LLMProxy.Config.Catalog.md): Boundary parser for catalog application configuration.
- [LLMProxy.Config.Provider](LLMProxy.Config.Provider.md): Release config provider for optional standalone LLMProxy TOML config.
- [LLMProxy.Config.TOML](LLMProxy.Config.TOML.md): TOML decoder for standalone LLMProxy configuration files.
- [LLMProxy.GuardrailPipeline](LLMProxy.GuardrailPipeline.md): Runs configured `LLMProxy.Guardrail` callbacks around requests, responses, and stream events.

- [LLMProxy.HTTP](LLMProxy.HTTP.md): Req client factory that applies LLMProxy's HTTP instrumentation and test plug configuration.

- [LLMProxy.HTTP.RouteSpec](LLMProxy.HTTP.RouteSpec.md): Route table shared by standalone Plug routing and Phoenix router embedding macros.

- [LLMProxy.HTTP.Routes.Chat](LLMProxy.HTTP.Routes.Chat.md): OpenAI Chat Completions route adapter for authenticated HTTP requests and SSE streams.

- [LLMProxy.HTTP.Routes.Dynamic](LLMProxy.HTTP.Routes.Dynamic.md): Registry for dynamically registered routes from optional packages.
- [LLMProxy.HTTP.Routes.Feedback](LLMProxy.HTTP.Routes.Feedback.md): Trace feedback endpoint for attaching user ratings and comments to request IDs or trace IDs.

- [LLMProxy.HTTP.Routes.MessageEndpoint](LLMProxy.HTTP.Routes.MessageEndpoint.md): Anthropic Messages route adapter that preserves provider-native request and response shapes.

- [LLMProxy.HTTP.Routes.ModelEndpoint](LLMProxy.HTTP.Routes.ModelEndpoint.md): Serves the OpenAI-compatible model list endpoint.

- [LLMProxy.HTTP.Routes.ModerationEndpoint](LLMProxy.HTTP.Routes.ModerationEndpoint.md): Serves OpenAI-compatible moderation requests.

- [LLMProxy.HTTP.Routes.ModerationEndpoint.CreateRequest](LLMProxy.HTTP.Routes.ModerationEndpoint.CreateRequest.md): Parsed moderation request body.

- [LLMProxy.HTTP.Routes.Passthrough](LLMProxy.HTTP.Routes.Passthrough.md): Shared provider-passthrough route flow.
- [LLMProxy.HTTP.Routes.Passthrough.ErrorHandler](LLMProxy.HTTP.Routes.Passthrough.ErrorHandler.md): Route callbacks used to render and classify passthrough-route errors.

- [LLMProxy.HTTP.Routes.Passthrough.ResultHandler](LLMProxy.HTTP.Routes.Passthrough.ResultHandler.md): Route callbacks used to render passthrough non-streaming and streaming results.

- [LLMProxy.HTTP.Routes.ResponseEndpoint](LLMProxy.HTTP.Routes.ResponseEndpoint.md): OpenAI Responses route adapter that passes native request and stream events through LLMProxy execution.

- [LLMProxy.HTTP.Routes.Setup](LLMProxy.HTTP.Routes.Setup.md): Optional setup-helper routes for install scripts, model lists, and client configuration snippets.

- [LLMProxy.HTTP.Routes.Setup.Auth](LLMProxy.HTTP.Routes.Setup.Auth.md): Parsed setup endpoint authentication.

- [LLMProxy.Limit](LLMProxy.Limit.md): Usage limit for API keys.

- [LLMProxy.ModelDB](LLMProxy.ModelDB.md): LLMDB integration for provider model lists and pricing lookup used by routing and accounting.

- [LLMProxy.Plugs.Auth](LLMProxy.Plugs.Auth.md): Extracts and validates API key from Authorization header or x-api-key.
Assigns `:api_key` to the conn.

- [LLMProxy.Plugs.Drain](LLMProxy.Plugs.Drain.md): Rejects new user work while LLMProxy is draining.
- [LLMProxy.Plugs.JSONBodyParser](LLMProxy.Plugs.JSONBodyParser.md): Parses authenticated JSON request bodies with a bounded limit and a structured oversized response.
- [LLMProxy.Plugs.QuotaCheck](LLMProxy.Plugs.QuotaCheck.md): Checks token/message/cache quotas for the authenticated API key.
Must run after LLMProxy.Plugs.Auth.

- [LLMProxy.Pricing](LLMProxy.Pricing.md): Model pricing lookup and cost calculation.
- [LLMProxy.Pricing.Rates](LLMProxy.Pricing.Rates.md): Per-million-token pricing rates used by LLMProxy cost accounting.
- [LLMProxy.Protocol](LLMProxy.Protocol.md): Protocol detection and conversion between LLM API formats.
- [LLMProxy.Protocol.Anthropic](LLMProxy.Protocol.Anthropic.md): Anthropic Messages API protocol.
- [LLMProxy.Protocol.OpenAI](LLMProxy.Protocol.OpenAI.md): OpenAI chat completion protocol.
- [LLMProxy.Protocol.Request](LLMProxy.Protocol.Request.md): Boundary parser that normalizes OpenAI Chat, Anthropic Messages, and OpenAI Responses requests.

- [LLMProxy.Protocol.Request.Error](LLMProxy.Protocol.Request.Error.md): Structured request-parse error returned by protocol request boundary parsers.

- [LLMProxy.Providers.Anthropic](LLMProxy.Providers.Anthropic.md): Built-in Anthropic Messages provider for native and OpenAI-compatible LLMProxy execution.

- [LLMProxy.Providers.Attempt](LLMProxy.Providers.Attempt.md): Normalized provider deployment attempt used by routing, fallback, cache keys, and circuit breakers.

- [LLMProxy.Providers.Behaviour](LLMProxy.Providers.Behaviour.md): Behaviour for LLM provider implementations.
- [LLMProxy.Providers.CircuitBreaker](LLMProxy.Providers.CircuitBreaker.md): GenServer-backed deployment circuit breaker for routing skips, half-open probes, and cooldowns.

- [LLMProxy.Providers.HTTPResult](LLMProxy.Providers.HTTPResult.md): Converts upstream HTTP responses and exceptions into `LLMProxy.Providers.Result` values.

- [LLMProxy.Providers.OpenAI](LLMProxy.Providers.OpenAI.md): Built-in OpenAI provider backed by the shared OpenAI-compatible provider implementation.

- [LLMProxy.Providers.OpenAICodex](LLMProxy.Providers.OpenAICodex.md): Adapter for the ChatGPT Codex backend exposed by ReqLLM.
- [LLMProxy.Providers.OpenAICodex.Events](LLMProxy.Providers.OpenAICodex.Events.md): Converts ReqLLM Codex stream chunks into LLMProxy stream events.
- [LLMProxy.Providers.OpenAICodex.OAuth](LLMProxy.Providers.OpenAICodex.OAuth.md): OAuth credential refresh boundary for the OpenAI Codex provider.
- [LLMProxy.Providers.OpenAICodex.OAuth.Login](LLMProxy.Providers.OpenAICodex.OAuth.Login.md): Interactive OpenAI Codex OAuth login for standalone LLMProxy releases.
- [LLMProxy.Providers.OpenAICodex.OAuth.Login.AccessTokenClaims](LLMProxy.Providers.OpenAICodex.OAuth.Login.AccessTokenClaims.md): Codex access-token JWT claims used by LLMProxy.
- [LLMProxy.Providers.OpenAICodex.OAuth.Login.AuthClaims](LLMProxy.Providers.OpenAICodex.OAuth.Login.AuthClaims.md): OpenAI auth claim payload embedded in the Codex access-token JWT.
- [LLMProxy.Providers.OpenAICodex.OAuth.Login.TokenResponse](LLMProxy.Providers.OpenAICodex.OAuth.Login.TokenResponse.md): OpenAI OAuth token endpoint response.
- [LLMProxy.Providers.OpenAICodex.OAuth.RefreshResponse](LLMProxy.Providers.OpenAICodex.OAuth.RefreshResponse.md): ReqLLM OpenAI Codex OAuth refresh response.
- [LLMProxy.Providers.OpenAICompatible](LLMProxy.Providers.OpenAICompatible.md): Shared HTTP client implementation for upstream providers with OpenAI-compatible chat endpoints.

- [LLMProxy.Providers.OpenAICompatible.Definition](LLMProxy.Providers.OpenAICompatible.Definition.md): Macro frontend for defining OpenAI-compatible upstream providers.
- [LLMProxy.Providers.OpenRouter](LLMProxy.Providers.OpenRouter.md): Built-in OpenRouter provider backed by the shared OpenAI-compatible provider implementation.

- [LLMProxy.Providers.ReqLLM](LLMProxy.Providers.ReqLLM.md): Configuration-driven upstream provider backed by ReqLLM.
- [LLMProxy.Providers.Result](LLMProxy.Providers.Result.md): Provider execution result tagged by response kind.
- [LLMProxy.Providers.Routing.RoundRobin](LLMProxy.Providers.Routing.RoundRobin.md): GenServer-backed routing strategy that rotates deployments within each order group by model name.

- [LLMProxy.RPC.AdminServer](LLMProxy.RPC.AdminServer.md): SafeRPC socket server for LLMProxy local RPC surfaces.
- [LLMProxy.Schemas.ApiKey](LLMProxy.Schemas.ApiKey.md): Ecto schema for LLMProxy API keys, quotas, budgets, model access, and aggregate usage counters.

- [LLMProxy.Schemas.MessageLog](LLMProxy.Schemas.MessageLog.md): Ecto schema for captured user-message snippets used by operations and admin views.

- [LLMProxy.Schemas.ProviderToken](LLMProxy.Schemas.ProviderToken.md): Ecto schema for upstream provider API keys, OAuth tokens, labels, proxies, and enabled state.

- [LLMProxy.Schemas.ServiceUsage](LLMProxy.Schemas.ServiceUsage.md): Ecto schema for non-provider service usage counters tied to an API key and endpoint.

- [LLMProxy.Schemas.Trace](LLMProxy.Schemas.Trace.md): Ecto schema for optional traced request and response payloads with usage and latency metadata.

- [LLMProxy.Schemas.TraceFeedback](LLMProxy.Schemas.TraceFeedback.md): Ecto schema for user feedback associated with traces or request IDs.

- [LLMProxy.Schemas.UsageLog](LLMProxy.Schemas.UsageLog.md): Ecto schema for per-request token, cost, latency, provider, tag, and metadata usage records.

- [LLMProxy.Storage.Ecto](LLMProxy.Storage.Ecto.md): Context functions for all database operations: keys, usage, quotas, tokens.

- [LLMProxy.Storage.JSON](LLMProxy.Storage.JSON.md): Ecto type for storing map and list values as JSON strings across supported SQL adapters.

- [LLMProxy.Storage.QuackDBServer](LLMProxy.Storage.QuackDBServer.md): Starts the bundled QuackDB server and wires its generated token into the bundled Ecto repo.
- [LLMProxy.Storage.Repo](LLMProxy.Storage.Repo.md): Facade for the configured LLMProxy Ecto repo.
- [LLMProxy.Storage.Repo.QuackDB](LLMProxy.Storage.Repo.QuackDB.md): Bundled QuackDB Ecto repo for production DuckDB-backed storage.

- [LLMProxy.Storage.Repo.SQLite](LLMProxy.Storage.Repo.SQLite.md): Bundled SQLite Ecto repo for local development, tests, and standalone storage.

- [LLMProxy.Storage.SQL](LLMProxy.Storage.SQL.md): SQL adapter detection and support checks for storage queries and migrations.

- [LLMProxy.Stream.Event](LLMProxy.Stream.Event.md): Server-sent stream event with protocol wire-event constructors.

- [LLMProxy.Stream.Heartbeat](LLMProxy.Stream.Heartbeat.md): Adds periodic heartbeat markers while an upstream enumerable is silent.
- [LLMProxy.Stream.SSEWriter](LLMProxy.Stream.SSEWriter.md): Helpers for writing SSE events to a Plug.Conn chunked response.

- [LLMProxy.TokenPool.Server](LLMProxy.TokenPool.Server.md): Manages provider tokens with rate-limit cooldowns.
- [LLMProxy.TokenPool.Server.State](LLMProxy.TokenPool.Server.State.md): Runtime state for provider-token cooldown timestamps.

- [LLMProxy.Trace](LLMProxy.Trace.md): Request and trace ID generation plus Plug response-header propagation.

- Core
  - [LLMProxy](LLMProxy.md): Elixir-native LLM gateway for in-process, ReqLLM, SafeRPC, and HTTP calls.
  - [LLMProxy.Actor](LLMProxy.Actor.md): Authenticated caller identity used below transport boundaries.

  - [LLMProxy.Provider](LLMProxy.Provider.md): Core in-process LLMProxy provider and ReqLLM adapter.
  - [LLMProxy.Response](LLMProxy.Response.md): Internal non-stream LLMProxy response.
  - [LLMProxy.Usage](LLMProxy.Usage.md): Token usage accounting and protocol usage-map rendering.

- Catalog and Routing
  - [LLMProxy.Catalog](LLMProxy.Catalog.md): Public model catalog for aliases and deployment metadata.

  - [LLMProxy.Catalog.Deployment](LLMProxy.Catalog.Deployment.md): Internal catalog deployment target for a public model alias.

  - [LLMProxy.Catalog.Model](LLMProxy.Catalog.Model.md): Internal catalog entry for a public model alias.

  - [LLMProxy.Providers.Execution](LLMProxy.Providers.Execution.md): Executes provider attempts with timeout, retry, fallback, telemetry, and circuit-breaker handling.
  - [LLMProxy.Providers.Registry](LLMProxy.Providers.Registry.md): Provider discovery via :persistent_term.

- Extensions
  - [LLMProxy.Cache](LLMProxy.Cache.md): Behaviour for deterministic response cache adapters.
  - [LLMProxy.Guardrail](LLMProxy.Guardrail.md): Behaviour for host-defined request/response policy hooks.
  - [LLMProxy.Storage](LLMProxy.Storage.md): Public persistence facade for keys, usage, quotas, tokens, traces, and feedback.

  - [LLMProxy.Storage.Adapter](LLMProxy.Storage.Adapter.md): Behaviour for persistence adapters that back LLMProxy storage operations.

- HTTP and Phoenix
  - [LLMProxy.HTTP.Router](LLMProxy.HTTP.Router.md): Standalone Plug router for LLMProxy health checks, core API routes, setup routes, and dynamic extensions.

  - [LLMProxy.Phoenix.Router](LLMProxy.Phoenix.Router.md): Phoenix router macros that mount LLMProxy route groups inside host applications.

  - [LLMProxy.Router](LLMProxy.Router.md): Public router facade for embedding LLMProxy.

- Operations
  - [LLMProxy.Drain](LLMProxy.Drain.md): Runtime drain control for deployment-safe LLMProxy shutdowns.
  - [LLMProxy.Ops](LLMProxy.Ops.md): Local operational RPC surface for deployment lifecycle control.
  - [LLMProxy.ReleaseTasks](LLMProxy.ReleaseTasks.md): Release-safe operational tasks for standalone LLMProxy deployments.
  - [LLMProxy.Telemetry](LLMProxy.Telemetry.md): Telemetry and OpenTelemetry helpers for provider routing and execution spans.

## Mix Tasks

- [mix llm_proxy.install](Mix.Tasks.LlmProxy.Install.md): Installs LLMProxy embedder migration aliases into an existing Mix project.

