ExAthena.Config (ExAthena v0.17.0)

Copy Markdown View Source

Resolves the provider and per-call options for an ExAthena request.

Resolution order (per key, highest to lowest priority):

  1. opts[:key] — per-call override always wins.
  2. For atom-named providers: Application.get_env(:ex_athena, provider)[:key] — provider-specific config.exs entry. For string-named providers: the matching JSON file loaded from ~/.config/ex_athena/providers/ by ExAthena.ProviderRegistry.
  3. Application.get_env(:ex_athena, :key) — top-level library config.
  4. Provider default (if the provider declares one).

Matches the stripity_stripe / ex_aws pattern: per-call overrides win, application config is the default, no global mutable state.

String provider names (e.g. provider: "my-groq") are resolved through ExAthena.ProviderRegistry, which loads *.json files from ~/.config/ex_athena/providers/ at application startup. See the Providers guide for the full JSON schema, security notes, and ready-to-copy examples.

Known providers

AtomModule
:ollamaExAthena.Providers.ReqLLM
:openai_compatibleExAthena.Providers.ReqLLM
:openaiExAthena.Providers.ReqLLM
:llamacppExAthena.Providers.ReqLLM
:claudeExAthena.Providers.ReqLLM
:anthropicExAthena.Providers.ReqLLM
:geminiExAthena.Providers.ReqLLM
:openrouterExAthena.Providers.ReqLLM
:req_llmExAthena.Providers.ReqLLM
:mockExAthena.Providers.Mock

You may also pass any module that implements ExAthena.Provider directly, or define custom providers by placing JSON files in ~/.config/ex_athena/providers/ (loaded at startup by ExAthena.ProviderRegistry).

Request queue

ExAthena.RequestQueue is an opt-in semaphore that caps concurrent in-flight requests per provider. Enable it with:

config :ex_athena, :request_queue, enabled: true

Per-provider depth limits can be set inside provider config blocks:

config :ex_athena, :ollama, request_queue: [max_depth: 3]

A global max_depth applies to all providers that don't have a per-provider override:

config :ex_athena, :request_queue, enabled: true, max_depth: 5

Built-in defaults (used when no application config is set):

ProviderDefault max_depth
:ollama1
:llamacpp1
:exo1
:openai, :anthropic, :claude, :gemini, :openrouter, :req_llm10
unknown10

Local providers deliberately default to a single slot until the serving setup is load-tested — raise at runtime via set_request_queue_max_depth/2 (the hosts expose this next to the provider selector).

Summary

Functions

Look up a single configuration value for provider_module with the tiered resolution order. opts wins, then provider-specific config, then top-level config, then the supplied default.

Return all known providers: built-in atoms plus any specs loaded by ExAthena.ProviderRegistry.

Pop :provider from opts and return {provider_module, remaining_opts}.

Resolve a provider atom (or module) to its implementing module.

Build the keyword list passed to a provider's query/2 / stream/3 callback.

Resolve a provider's ProviderSpec. A user-registered JSON spec (loaded by ExAthena.ProviderRegistry) wins; otherwise a built-in spec (e.g. OpenRouter) is returned. Returns :error when neither exists.

Translate an ExAthena provider atom into the req_llm provider tag used in "tag:model-id" specs. Returns nil when the atom doesn't map to req_llm (e.g. :mock, or a user-supplied module).

Return true when the request queue feature is enabled.

Return the maximum concurrent request depth for provider_atom.

Set the per-provider concurrent-slot cap at runtime.

Functions

get(provider_module, key, opts, default \\ nil)

@spec get(module(), atom(), keyword(), term()) :: term()

Look up a single configuration value for provider_module with the tiered resolution order. opts wins, then provider-specific config, then top-level config, then the supplied default.

The provider atom is derived from the module name: Providers.Ollama:ollama.

list_providers()

@spec list_providers() :: [
  %{
    name: String.t(),
    display_name: String.t(),
    module: module(),
    source: :builtin | :registry
  }
]

Return all known providers: built-in atoms plus any specs loaded by ExAthena.ProviderRegistry.

Each entry is a map with:

  • :name — string name of the provider
  • :module — the implementing module
  • :source:builtin or :registry

pop_provider!(opts)

@spec pop_provider!(keyword()) :: {module(), keyword()}

Pop :provider from opts and return {provider_module, remaining_opts}.

Raises ArgumentError if no provider is set in opts or in application env.

For registry-loaded providers the following spec fields are threaded into opts using Keyword.put_new/3 so per-call overrides always win:

  • :req_llm_provider_tag — from spec.req_llm_provider_tag
  • :base_url — from spec.base_url
  • :extra_headers — from spec.extra_headers (non-empty maps only)
  • :api_key — from spec.api_key, or resolved from spec.api_key_env

provider_module(mod)

@spec provider_module(atom() | module()) :: module()

Resolve a provider atom (or module) to its implementing module.

Resolution order:

  1. Built-in provider map (compile-time constant, fastest path).
  2. ExAthena.ProviderRegistry — for providers loaded from JSON config files.
  3. Module check — accepts any atom that is a loaded module implementing ExAthena.Provider.

provider_opts(provider_module, opts, provider_atom \\ nil)

@spec provider_opts(module(), keyword(), atom() | nil) :: keyword()

Build the keyword list passed to a provider's query/2 / stream/3 callback.

Flattens per-call overrides + application env for this provider into one keyword list. Providers use Keyword.get/3 on the result.

provider_spec(provider)

@spec provider_spec(atom() | String.t()) :: {:ok, ExAthena.ProviderSpec.t()} | :error

Resolve a provider's ProviderSpec. A user-registered JSON spec (loaded by ExAthena.ProviderRegistry) wins; otherwise a built-in spec (e.g. OpenRouter) is returned. Returns :error when neither exists.

req_llm_provider_tag(atom)

@spec req_llm_provider_tag(atom() | module()) :: String.t() | nil

Translate an ExAthena provider atom into the req_llm provider tag used in "tag:model-id" specs. Returns nil when the atom doesn't map to req_llm (e.g. :mock, or a user-supplied module).

Resolution order: a registry-loaded JSON spec wins (its req_llm_provider_tag field), with the static built-in map as fallback when no spec is loaded for the atom or the spec's tag is unset. This matters when a JSON-defined provider's name collides with a built-in atom (e.g. :openrouter, which is also present in the built-in map for config.exs users) — the user's JSON file is authoritative.

request_queue_enabled?()

@spec request_queue_enabled?() :: boolean()

Return true when the request queue feature is enabled.

Enabled by default — local inference servers (ollama / llama.cpp / exo) can only serve 1-3 concurrent requests, and the per-call gate is what keeps concurrent loops and subagents from overwhelming them. Disable with:

config :ex_athena, :request_queue, enabled: false

request_queue_max_depth(provider_atom)

@spec request_queue_max_depth(atom()) :: pos_integer()

Return the maximum concurrent request depth for provider_atom.

Resolution order:

  1. config :ex_athena, provider_atom, request_queue: [max_depth: N]
  2. config :ex_athena, :request_queue, max_depth: N
  3. Built-in per-provider default (local providers: 1, cloud: 10).
  4. 10 for unrecognised providers.

set_request_queue_max_depth(provider_atom, depth)

@spec set_request_queue_max_depth(atom(), pos_integer()) ::
  :ok | {:error, :invalid_depth}

Set the per-provider concurrent-slot cap at runtime.

Hosts expose this next to their provider selector so users can raise a local provider's slots once their serving setup proves it can take more load (everything local defaults to 1). Merges into the provider's existing application env, preserving keys like base_url.

Takes effect on the next slot acquisition — in-flight holders and waiters are unaffected.