AgentSea.Gateway (agentsea_gateway v0.1.0)

Copy Markdown

Routes completion requests across multiple providers with strategy-based ordering, circuit breaking, and failover.

A request is planned (the strategy orders the available candidates, blown fuses and excluded providers removed) and then tried in order in the caller's process — so requests stay concurrent. Each attempt records latency and, on failure, melts the provider's fuse. When every candidate is exhausted the gateway returns {:error, :all_providers_unavailable}.

Example

providers = [
  %AgentSea.Gateway.Provider{name: :primary, module: MyAnthropic, model: "claude-opus-4-8"},
  %AgentSea.Gateway.Provider{name: :backup,  module: MyOpenAI,    model: "gpt-4.1"}
]

{:ok, gw} = AgentSea.Gateway.start_link(%AgentSea.Gateway.Config{
  providers: providers,
  strategy: AgentSea.Gateway.Router.Failover
})

{:ok, response, served_by} = AgentSea.Gateway.completion(gw, messages)

Summary

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

completion(gateway, messages, opts \\ [])

Route a completion. Options: :exclude (provider names to skip). Returns {:ok, response, provider_name} or {:error, :all_providers_unavailable}.

health(gateway)

Current per-provider health (latency EMA + call/error counts).

start_link(config, opts \\ [])
stream(gateway, messages, opts \\ [])

Route a streaming completion. Picks the first available provider (in strategy order) that implements AgentSea.Provider.stream/2 and returns its lazy event stream. There is no mid-stream failover (a stream can't be replayed).

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

completion(gateway, messages, opts \\ [])

@spec completion(GenServer.server(), [AgentSea.Provider.message()], keyword()) ::
  {:ok, AgentSea.Response.t(), term()} | {:error, :all_providers_unavailable}

Route a completion. Options: :exclude (provider names to skip). Returns {:ok, response, provider_name} or {:error, :all_providers_unavailable}.

health(gateway)

Current per-provider health (latency EMA + call/error counts).

start_link(config, opts \\ [])

stream(gateway, messages, opts \\ [])

@spec stream(GenServer.server(), [AgentSea.Provider.message()], keyword()) ::
  {:ok, Enumerable.t(), term()} | {:error, term()}

Route a streaming completion. Picks the first available provider (in strategy order) that implements AgentSea.Provider.stream/2 and returns its lazy event stream. There is no mid-stream failover (a stream can't be replayed).

Returns {:ok, stream, provider_name} or an error.