LLMProxy.TokenPool.Server (llm_proxy v0.2.0)

Manages provider tokens with rate-limit cooldowns.

Picks tokens using the configured affinity or fill-first strategy. Priority: OAuth tokens first, then API keys as fallback.

Summary

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

clear_rate_limits()

mark_rate_limited(token, cooldown_ms \\ LLMProxy.Config.token_cooldown_ms())

mark_rate_limited(arg1, model, cooldown_ms)

pick_token(provider, user_id \\ "", model \\ nil)

pick_token_by_kind(provider, kind, user_id \\ "", model \\ nil)

start_link(opts)

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

clear_rate_limits()

mark_rate_limited(token, cooldown_ms \\ LLMProxy.Config.token_cooldown_ms())

mark_rate_limited(arg1, model, cooldown_ms)

pick_token(provider, user_id \\ "", model \\ nil)

pick_token_by_kind(provider, kind, user_id \\ "", model \\ nil)

start_link(opts)