LlmCore.LLM.Native.Router (llm_core v0.3.0)

Copy Markdown View Source

Config-driven provider resolution for the Native agentic loop.

Resolves a provider name (or nil for cascade) into a fully-configured {module, model, provider_opts} tuple. Provider definitions are loaded from TOML at startup and stored in LlmCore.Config.Store.

Resolution Logic

  1. Explicit provider (llm_provider: "zai") → look up Definition by name/alias
  2. Model routing (model matches a pattern) → look up Definition for matched provider
  3. Cascade → walk [native]cascade list, find first available provider Definition
  4. Nothing matches → {:error, :no_provider}

Adding a new provider

Add a [providers.my_provider] section to llm_core.toml. Zero code changes.

Summary

Functions

Returns an ordered list of candidates — primary first, then cascade fallbacks.

Walk the cascade and return the first available provider.

Returns the default model for a provider alias from config.

Resolve a provider for the given model string.

Resolve a provider by explicit name (e.g. "zai" from native:zai syntax).

Match a lowercased model string against routing patterns. First match wins.

Types

config()

@type config() :: map()

provider_opts()

@type provider_opts() :: keyword()

resolution()

@type resolution() ::
  {:ok, {module(), String.t(), provider_opts()}} | {:error, :no_provider}

Functions

candidates(model, config, opts \\ [])

@spec candidates(String.t() | nil, config(), keyword()) :: [
  {module(), String.t(), provider_opts()}
]

Returns an ordered list of candidates — primary first, then cascade fallbacks.

The primary is what resolve/3 would pick. Fallbacks are the remaining cascade providers with their own default models (NOT the originally requested model — a claude-named model can't run on Appliance).

The primary module is never duplicated in the tail. Callers walk the list and attempt each candidate in order, falling through on runtime failure.

Returns [] when nothing resolves.

cascade_pick(config, model, providers)

@spec cascade_pick(config(), String.t() | nil, map()) :: resolution()

Walk the cascade and return the first available provider.

get_default_model(alias, config)

@spec get_default_model(String.t(), config()) :: String.t() | nil

Returns the default model for a provider alias from config.

resolve(model, config, opts \\ [])

@spec resolve(String.t() | nil, config(), keyword()) :: resolution()

Resolve a provider for the given model string.

Returns {:ok, {module, model, provider_opts}} or {:error, :no_provider}.

The appliance_has_model flag lets the caller indicate whether the model is available locally (checked externally via Appliance discovery).

resolve_provider(provider_name)

@spec resolve_provider(String.t()) :: resolution()

Resolve a provider by explicit name (e.g. "zai" from native:zai syntax).

This is the direct routing path — no cascade, no model routing. Returns {:ok, {module, model, provider_opts}} or {:error, :no_provider}.

route_model(lower, config)

@spec route_model(String.t(), config()) :: {:ok, String.t()} | :no_match

Match a lowercased model string against routing patterns. First match wins.