# Configuration

Library mode uses ordinary Elixir application configuration. Standalone mode reads non-secret runtime and routing data from strict TOML, while environment variables are reserved for secrets and the TOML file location.

## Application Options
{: .col-2}

### Repo

```elixir
config :llm_proxy, repo: MyApp.Repo
```

Ecto repo used by the default storage adapter. Library hosts normally provide an already supervised repo. Standalone production uses `LLMProxy.Storage.Repo.QuackDB`.

### Storage Adapter

```elixir
config :llm_proxy, storage: MyApp.LLMProxyStorage
```

Module implementing `LLMProxy.Storage.Adapter`. Default: `LLMProxy.Storage.Ecto`.

### HTTP Enabled

```elixir
config :llm_proxy, http_enabled: false
```

Whether LLMProxy starts its own loopback Cowboy listener. Default: `true`. Disable in library mode unless a separate listener is intentional.

### HTTP Port

```elixir
config :llm_proxy, http: [port: 4000]
```

Port for the bundled listener. The listener binds `127.0.0.1`.

### Master Key

```elixir
config :llm_proxy, master_key: System.fetch_env!("LLM_PROXY_MASTER_KEY")
```

Bootstrap/operator credential. It bypasses ordinary model and quota checks.

### RPC Socket

```elixir
config :llm_proxy, rpc_socket: "/run/llm-proxy/rpc.sock"
```

Starts the [SafeRPC](https://hexdocs.pm/safe_rpc) server for chat, operations, and optional Incant admin calls.

### Public URL

```elixir
config :llm_proxy, public_url: "https://llm.example.com"
```

Public base URL used by setup helpers and provider headers.

### Request Body Limit

```elixir
config :llm_proxy, body_limit_bytes: 32_000_000
```

Maximum authenticated JSON request body. Must be a positive integer. Default: 32 MB.

### Remote Timeout

```elixir
config :llm_proxy, remote_timeout_ms: 30_000
```

Default SafeRPC request timeout. Default: 30 seconds.

### Provider Receive Timeout

```elixir
config :llm_proxy, provider_receive_timeout_ms: :timer.minutes(10)
```

Provider transport timeout and standalone Cowboy idle ceiling. Default: 10 minutes.

### Token Cooldown

```elixir
config :llm_proxy, token_cooldown_ms: :timer.hours(4)
```

Default credential cooldown after rate limiting. Must be a positive integer no greater than 31
days. Default: 4 hours.

### Provider Token Selection

```elixir
config :llm_proxy, token_selection_strategy: :fill_first
```

`:affinity` is the default and preserves stable user-to-token affinity.
`:fill_first` selects the highest-priority healthy token within the preferred
credential kind, breaking ties by token ID. OAuth remains preferred over API
keys. Standalone TOML:

```toml
[provider_tokens]
selection_strategy = "fill_first"
```

This non-secret setting is not read from the environment. Provider-token
priorities are non-negative integers and can be edited through Incant's
priority-only form. Set an initial priority when adding a token:

```elixir
LLMProxy.Storage.add_token("openai-codex", "oauth", access_token, %{priority: 100})
```

### Provider Token Codec

```elixir
config :llm_proxy,
  provider_token_codec: {
    LLMProxy.Provider.TokenCodec.AESGCM,
    active_key_id: "2026-08",
    keys: %{
      "2026-08" => System.fetch_env!("PROVIDER_TOKEN_KEY_2026_08"),
      "2026-02" => System.fetch_env!("PROVIDER_TOKEN_KEY_2026_02")
    },
    allow_plaintext: true
  }
```

Codec for provider API keys and OAuth tokens. AES-GCM keys must decode to 32
bytes. This keyring is separate from `:master_key`. Writes use only the active
key ID. Reads accept all key IDs in the keyring. The default plaintext codec is
for compatibility and does not protect stored credentials.

### Deployment Failure Threshold

```elixir
config :llm_proxy, deployment_failure_threshold: 3
```

Default consecutive retryable failures before opening a deployment circuit. Default: 3.

### Deployment Cooldown

```elixir
config :llm_proxy, deployment_cooldown_ms: 30_000
```

Default open-circuit cooldown. Default: 30 seconds.

### Max Retries

```elixir
config :llm_proxy, max_retries: 1
```

Compatibility retry/fallback limit for configured fallback chains. Default: 1. The strict provider-dispatch budget is `max_retries + 1`; open-circuit and unsupported-protocol route skips do not consume it.

Standalone TOML:

```toml
[routing]
max_retries = 1
```

### Replay Policy

```elixir
config :llm_proxy, replay_policy: :safe_only
```

`:safe_only` permits fallback only when LLMProxy has evidence that the upstream did not accept work, such as unavailable credentials, a clear connection failure, or a 429 refusal. This is the default.

`:allow_uncertain` also permits replay after timeouts and 5xx responses. Use it only when duplicate cost and side effects are acceptable. It restores the earlier fallback behavior.

Standalone TOML:

```toml
[routing]
replay_policy = "safe_only"
```

### Fallbacks

```elixir
config :llm_proxy,
  fallbacks: %{"primary" => ["secondary"]}
```

Compatibility map for model fallback chains. Catalog routes are preferred for new configurations.

### Cache Adapter

```elixir
config :llm_proxy, cache: MyApp.LLMCache
```

Module implementing `LLMProxy.Cache`. Unset by default.

### Cache Policy

```elixir
config :llm_proxy,
  cache_policy: [
    enabled: true,
    ttl_ms: 60_000,
    models: %{"fresh" => [enabled: false]}
  ]
```

Default and per-model deterministic cache policy.

### Guardrails

```elixir
config :llm_proxy, guardrails: [MyApp.RequestPolicy]
```

Ordered modules implementing `LLMProxy.Guardrail`.

## Provider Configuration
{: .col-2}

### Public Model Allowlist

```elixir
config :llm_proxy, public_models: ["fast", "codex"]
```

Filters model discovery and request admission. Unset by default, which exposes all registered models for compatibility. In standalone mode, configure visible catalog aliases in TOML; direct provider model IDs are not exposed by the allowlist:

```toml
[catalog]
public_models = ["fast", "codex"]
```

An explicit empty list exposes no models. This non-secret setting is not read from the environment.

### Built-in Provider Keys

```elixir
config :llm_proxy,
  providers: %{
    "openai" => %{api_keys: "sk-a,sk-b"},
    "anthropic" => %{api_keys: "sk-ant-..."},
    "openrouter" => %{api_keys: "sk-or-..."},
    "openai-codex" => %{oauth_tokens: "access|refresh|expires_ms|account"}
  }
```

Runtime bootstrap values are persisted into provider-token storage.

### Named Provider

```elixir
config :llm_proxy,
  providers: %{
    "example-service" => %{
      adapter: "openai",
      base_url: "https://api.example.com/v1",
      token_pool: "example-production"
    }
  }
```

Uses an existing ReqLLM adapter without a custom provider module.

### Provider Base URL

```elixir
%{base_url: "https://api.example.com/v1"}
```

Default endpoint for a named or built-in provider.

### Provider Token Pool

```elixir
%{token_pool: "example-production"}
```

Default credential pool for provider routes.

### GLM Usage Source

```elixir
%{
  adapter: "zai_coding_plan",
  base_url: "https://api.z.ai/api/coding/paas/v4",
  token_pool: "glm-production",
  usage_adapter: "glm",
  usage_auth_scheme: "raw",
  usage_paths: ["/api/monitor/usage/quota/limit", "/api/monitor/usage"]
}
```

`usage_adapter` is optional for `zai`, `zai_coder`, and `zai_coding_plan`; when present it must be the string `"glm"`. `usage_auth_scheme` must be `"raw"` or `"bearer"`. `usage_paths` must be a non-empty list of at most three distinct absolute origin paths. Raw whitespace and control bytes are rejected; encode valid path characters when needed. Invalid values fail configuration loading rather than being coerced or ignored.

### Anthropic Defaults

```elixir
%{
  api_version: "2023-06-01",
  beta: "feature-a,feature-b",
  conversion_defaults: %{max_tokens: 4096}
}
```

Provider-specific headers and request-conversion defaults.

## Model Configuration
{: .col-2}

### Single Route

```elixir
config :llm_proxy,
  models: [
    fast: [route: [to: :openai, model: "gpt-4.1-mini"]]
  ]
```

Public alias `fast` targets one upstream deployment.

### Multiple Routes

```elixir
config :llm_proxy,
  models: [
    fast: [
      routing: :ordered,
      routes: [
        [to: :openai, model: "gpt-4.1-mini"],
        [to: :anthropic, model: "claude-3-5-haiku-20241022", order: 2]
      ]
    ]
  ]
```

### Routing

```elixir
routing: :round_robin
```

`:ordered`, `:shuffle`, `:round_robin`, `:weighted_shuffle`, `:lowest_cost`, or
`:latency_aware`.

### Route Timeout

```elixir
timeout: 30_000
```

Per-attempt timeout in milliseconds. `timeout_ms` is also accepted.

### Route Circuit Breaker

```elixir
failure_threshold: 3,
cooldown_ms: 30_000
```

Overrides deployment defaults.

### Route Order

```elixir
order: 2
```

Fallback group. Lower groups are attempted first.

### Route Weight

```elixir
weight: 3
```

Relative weight for `:weighted_shuffle`.

### Route Token Pool

```elixir
token_pool: "special-production"
```

Overrides the provider's default pool.

## Standalone Environment
{: .col-2}

### Master Key

```text
MASTER_KEY=...
```

Bootstrap/operator credential.

### Provider API-Key Pools

```text
LLM_PROXY_PROVIDER_KEYS={"example-production":["secret"]}
```

Secret JSON object mapping token-pool names to API-key arrays. Use pool names
such as `openai`, `anthropic`, or `openrouter` for built-in providers. Codex OAuth
credentials are provisioned through the admin login flow and persisted in
provider-token storage.

### Provider Token Keyring

```text
LLM_PROXY_PROVIDER_TOKEN_KEYRING={"active_key_id":"2026-08","keys":{"2026-08":"base64-encoded-32-byte-key"}}
```

Secret JSON keyring for standalone provider-token encryption. Keys must decode
to 32 bytes. Keep this keyring separate from `MASTER_KEY`; writes use the active
key ID and reads accept every retained key.

### TOML Path

```text
LLM_PROXY_CONFIG_TOML=/etc/llm-proxy/config.toml
```

Standalone TOML data file. Default: `/etc/llm-proxy/config.toml`.

## Standalone TOML
{: .col-2}

### Server

```toml
[server]
port = 4000
public_url = "https://llm.example.com"
body_limit_bytes = 32000000
rpc_socket = "/run/llm-proxy/rpc.sock"
```

The listener remains bound to loopback. Omit `rpc_socket` to disable SafeRPC.

### Storage

```toml
[storage]
database = "/var/lib/llm-proxy/llm_proxy.duckdb"
quackdb_uri = "http://127.0.0.1:9494"
quackdb_endpoint = "quack:localhost:9494"
```

### Routing Runtime

```toml
[routing]
max_retries = 1
replay_policy = "safe_only"
provider_connect_timeout_ms = 10000
token_selection_strategy = "affinity"
```

Fallback deployments belong in model routes, not a separate fallback map.

### Provider Usage

```toml
[provider_usage]
auto_refresh = true
refresh_interval_ms = 300000
request_timeout_ms = 10000
stale_after_ms = 600000
```

The refresh interval must be from `60000` through `3600000`. The request timeout
must be from `1000` through `30000`. The stale time must be at least the refresh
interval and no more than `86400000`. Omitted values use the built-in defaults.

### Telemetry

```toml
[telemetry]
otlp_endpoint = "http://127.0.0.1:4318"
```

Omit the endpoint to keep trace export disabled.

### Provider Token Rollout

```toml
[provider_tokens]
allow_plaintext = true
```

Keep plaintext compatibility enabled only during an explicit encryption rollout.
After migration and verification, set it to `false`; a missing keyring then fails
startup.

### Provider

```toml
[providers.example-service]
adapter = "openai"
base_url = "https://api.example.com/v1"
token_pool = "example-production"
```

Optional provider usage keys are `usage_adapter`, `usage_auth_scheme`, and `usage_paths`.

### Model

```toml
[[models]]
name = "example/model"
routing = "ordered"
```

### Route

```toml
[[models.routes]]
to = "example-service"
model = "upstream-model"
timeout = 30000
failure_threshold = 3
cooldown_ms = 30000
order = 1
weight = 1
```

TOML contains data only. Unknown keys are rejected; arbitrary atoms are not created.