Architecture Overview

Copy Markdown

This document describes the high-level architecture of Sagents and how its components work together.

System Design Philosophy

Sagents is built on three core principles:

  1. OTP-Native: Every agent is a supervised GenServer process, leveraging Erlang/OTP's battle-tested concurrency primitives
  2. Composable: Capabilities are added through middleware
  3. Observable: Real-time events flow directly from each agent to its subscribers via Sagents.Publisher (no Phoenix.PubSub topic in the path) for UI reactivity and debugging

Component Overview

┌─────────────────────────────────────────────────────────────────┐
│                        Your Application                         │
│  ┌──────────────┐   ┌──────────────┐   ┌──────────────────────┐ │
│  │  LiveView    │   │  Controller  │   │  Background Job      │ │
│  │  (ChatLive)  │   │              │   │                      │ │
│  └──────┬───────┘   └──────┬───────┘   └──────────┬───────────┘ │
└─────────┼──────────────────┼──────────────────────┼─────────────┘
          │                  │                      │
          │ AgentServer.subscribe(agent_id)         │
          │ (registers via Sagents.Publisher;       │
          │  events delivered to subscriber pids    │
          │  by direct send/2 — no PubSub topic)    │
          ▼                  ▼                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                        AgentSupervisor                          │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │                       AgentServer                           ││
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  ││
│  │  │    Agent    │  │    State    │  │  Middleware Stack   │  ││
│  │  │  (config)   │  │  (runtime)  │  │  [M1, M2, M3, ...]  │  ││
│  │  └─────────────┘  └─────────────┘  └─────────────────────┘  ││
│  └─────────────────────────────────────────────────────────────┘│
│  ┌────────────────────────────────────────────────────────────┐ │
│  │  SubAgentsDynamicSupervisor                                │ │
│  │  ┌───────────┐  ┌───────────┐                              │ │
│  │  │ SubAgent1 │  │ SubAgent2 │  ...                         │ │
│  │  └───────────┘  └───────────┘                              │ │
│  └────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
          │ references by scope
          ▼
┌─────────────────────────────────────────────────────────────────┐
│                   FileSystemSupervisor                          │
│  ┌─────────────────────┐  ┌─────────────────────┐               │
│  │  FileSystemServer   │  │  FileSystemServer   │  ...          │
│  │  ({:user, 1})       │  │  ({:project, 42})   │               │
│  └─────────────────────┘  └─────────────────────┘               │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                         LangChain                               │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐  │
│  │   LLMChain      │  │  ChatModels     │  │  Message        │  │
│  │  (execution)    │  │  (Anthropic,    │  │  ToolCall       │  │
│  │                 │  │   OpenAI, etc.) │  │  ToolResult     │  │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

Key Design Decision: FileSystemServer is supervised separately from AgentServer. This allows flexible scoping - for example, a project-scoped filesystem can be shared across multiple conversation-scoped agents. Agents reference filesystems by scope tuple (e.g., {:user, 123}, {:project, 456}), not by direct supervision.

Core Components

Agent

The Agent struct holds the configuration for an agent:

%Agent{
  agent_id: "conversation-123",
  model: %ChatAnthropic{...},
  base_system_prompt: "You are helpful.",
  middleware: [{TodoList, []}, {FileSystem, [enabled_tools: [...]]}, ...],
  tools: [custom_tool],  # Additional tools beyond middleware
  callbacks: %{...}      # Event callbacks
}

Key design decision: The Agent is immutable configuration. It doesn't hold runtime state - that's the State struct's job.

State

The State struct holds runtime data that changes during execution:

%State{
  agent_id: "conversation-123",
  messages: [%Message{...}, ...],
  todos: [%Todo{...}, ...],
  metadata: %{...},
  interrupt: nil | %InterruptData{...}
}

State flows through the middleware stack and accumulates:

  • Messages from user and LLM
  • Tool call results
  • TODO list updates
  • Middleware-specific metadata

AgentServer

The AgentServer is a GenServer that:

  1. Manages lifecycle - Starts, stops, handles timeouts
  2. Coordinates execution - Runs the middleware/LLM loop
  3. Broadcasts events - Delivers to subscriber pids via Sagents.Publisher
  4. Handles interrupts - Pauses for HITL (Human In The Loop) and resumes
# Simplified execution loop
def handle_cast(:execute, state) do
  case execute_agent_loop(state) do
    {:ok, new_state} ->
      broadcast(:status_changed, :idle, nil)
      {:noreply, %{state | agent_state: new_state}}

    {:interrupt, new_state, interrupt_data} ->
      broadcast(:status_changed, :interrupted, interrupt_data)
      {:noreply, %{state | agent_state: new_state, interrupt: interrupt_data}}

    {:error, reason} ->
      broadcast(:status_changed, :error, reason)
      {:noreply, state}
  end
end

Middleware

Middleware implements the Sagents.Middleware behaviour:

@callback init(opts :: keyword()) :: {:ok, config :: map()} | {:error, reason}
@callback system_prompt(config) :: String.t() | nil
@callback tools(config) :: [Function.t()]
@callback before_model(state, config) :: {:ok, state} | {:interrupt, state, data}
@callback after_model(state, config) :: {:ok, state} | {:interrupt, state, data}
@callback handle_message(message, state, config) :: {:ok, state}
@callback on_server_start(state, config) :: {:ok, state}

Middleware is applied in order:

  • before_model: First middleware runs first
  • after_model: First middleware runs last (reversed order)

This creates a "sandwich" pattern where early middleware wraps later middleware.

Data Flow

Message Execution Flow

User sends message
        │
        ▼
┌───────────────────────────────────────┐
│  AgentServer.add_message/2            │
│  Triggers execute/1                   │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  Middleware: before_model (in order)  │
│  - TodoList: No-op                    │
│  - Summarization: Check token count   │
│  - PatchToolCalls: Fix dangling calls │
│  - HITL: No-op (nothing to approve)   │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  Build LLMChain                       │
│  - System prompt (base + middleware)  │
│  - Messages from state                │
│  - Tools from middleware              │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  LLMChain.run (streaming)             │
│  - Deltas → broadcast                 │
│  - Tool calls → execute tools         │
│  - Complete message → broadcast       │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  Middleware: after_model (reverse)    │
│  - HITL: Check for protected calls    │  ← May INTERRUPT here
│  - PatchToolCalls: No-op              │
│  - Summarization: No-op               │
│  - TodoList: Broadcast todos          │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  Loop continues if needs_response?    │
│  (agent made tool calls)              │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  Execution complete                   │
│  - Status → :idle                     │
│  - State persisted (if configured)    │
└───────────────────────────────────────┘

Interrupt Flow (Human-In-The-Loop)

Agent makes protected tool call (e.g., write_file)
        │
        ▼
┌───────────────────────────────────────┐
│  HITL Middleware: after_model         │
│  Detects protected tool call          │
│  Returns {:interrupt, state, data}    │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  AgentServer stores interrupt         │
│  Broadcasts {:status_changed,         │
│              :interrupted, data}      │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  LiveView shows approval UI           │
│  User reviews tool calls              │
│  User makes decisions                 │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  AgentServer.resume(agent_id,         │
│                     decisions)        │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  HITL Middleware: apply_decisions     │
│  - :approve → Execute tool            │
│  - :edit → Execute with new args      │
│  - :reject → Return rejection msg     │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  Execution resumes from loop          │
└───────────────────────────────────────┘

SubAgent Flow

Parent agent calls spawn_subagent tool
        │
        ▼
┌───────────────────────────────────────┐
│  SubAgent Middleware creates child    │
│  - New AgentServer under              │
│    SubAgentsDynamicSupervisor         │
│  - Inherits HITL permissions          │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  SubAgent executes independently      │
│  - Own message history                │
│  - Own tool execution                 │
│  - Can also interrupt for HITL        │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  If SubAgent interrupts:              │
│  - Interrupt propagates to parent     │
│  - Parent shows approval UI           │
│  - Approval flows back to SubAgent    │
└───────────────────────────────────────┘
        │
        ▼
┌───────────────────────────────────────┐
│  SubAgent completes                   │
│  - Returns result to parent           │
│  - SubAgent process terminates        │
└───────────────────────────────────────┘

State Persistence

What Gets Persisted

# AgentState schema (serialized JSON)
%{
  "messages" => [...],           # Full message history
  "todos" => [...],              # Current TODO list
  "metadata" => %{               # Middleware state
    "conversation_title" => "Debug payment bug",
    "filesystem_files" => %{...}  # If using in-memory filesystem
  }
}

What Comes From Code

The Agent configuration is NOT persisted. This includes:

  • Model settings
  • Middleware stack
  • Tool definitions
  • System prompts

This separation means you can:

  • Update middleware without migrating stored data
  • A/B test different agent configurations
  • Keep secrets (API keys) out of the database

Restoration Pattern

Sagents.Session.start/3 (called by the generated Coordinator) handles this end-to-end: it consults the configured FactoryRouter, calls the factory with (agent_id, %FactoryConfig{}), loads persisted state via your AgentPersistence module, and starts the supervisor:

# Application code
{:ok, session} =
  Coordinator.start_conversation_session(conversation_id,
    scope: scope,
    request_opts: [timezone: "America/Denver"]
  )

# Internally (simplified)
{:ok, factory, config} = Router.resolve(scope, conversation_id, request_opts)
{:ok, agent, session_opts} = factory.create_agent(agent_id, config)
{:ok, state} = State.load_or_new(AgentPersistence, scope, %{...},
                                 fresh_state_attrs: session_opts[:fresh_state_attrs])
AgentsDynamicSupervisor.start_agent_sync(agent: agent, initial_state: state, ...)

The router is consulted on every start (including resume), so a restored conversation always rebuilds with the factory it was originally created with. :fresh_state_attrs is applied only when no persisted state is found; restored state always wins.

Registry and Discovery

Every agent process registers itself in Sagents.Registry at start, through the Sagents.ProcessRegistry abstraction. That module hides which backend is in play:

  • :local — Elixir's Registry (single node, no extra dependency)
  • :horde — Horde.Registry (cluster-wide, requires the :horde dependency)
config :sagents, :distribution, :local   # default
config :sagents, :distribution, :horde

Registration is by keyed :via tuple rather than by bare agent id, so a supervisor and the server inside it are separate entries:

Sagents.ProcessRegistry.via_tuple({:agent_server, "conversation-1"})
Sagents.ProcessRegistry.via_tuple({:agent_supervisor, "conversation-1"})

Discovery:

AgentServer.list_running_agents()
# => ["conversation-1", "conversation-2"]

AgentServer.fetch_pid("conversation-1")
# => {:ok, #PID<0.1234.0>}

A lookup has three outcomes

A registry read can only be answered while the registry process on this node is alive. It is not alive while the node is still starting Sagents.Supervisor, and it is not alive after that supervisor has shut down while the BEAM drains during a rolling deploy — a window that lasts as long as the platform's grace period, and during which a load balancer may still be routing requests here.

So the API keeps three answers distinct:

case AgentServer.fetch_pid(agent_id) do
  {:ok, pid} -> ...                        # running
  {:error, :not_running} -> ...            # the registry answered: nothing there
  {:error, :registry_unavailable} -> ...   # this node cannot answer at all
end

:registry_unavailable must never collapse into "not running". A caller that reads "nothing is running" responds by starting an agent, so collapsing the two lets a draining node start a second AgentServer for a conversation that already has one elsewhere. Both would hold and persist state for it, with nothing reporting the conflict.

Functions whose return shape cannot carry the condition raise Sagents.RegistryUnavailableError instead of answering nil, false, [] or 0 — AgentServer.get_pid/1, AgentServer.list_running_agents/0 and Sagents.Session.running?/2 among them. Prefer the tuple-returning forms (AgentServer.fetch_pid/1, Sagents.Session.ensure_running/3) anywhere a web request can reach, and map :registry_unavailable to a retryable 503 rather than a 500: the cluster can serve the request, just not on this node.

Sagents.ready?/0 exposes the same signal for a readiness endpoint. See Deployments, draining, and readiness.

Supervision Tree

Application Supervisor (your app)
│
├── MyApp.Repo
├── Phoenix.PubSub (:my_pubsub)
│
├── Sagents.Supervisor                          # strategy: :rest_for_one
│   │
│   ├── Sagents.Registry                        # Registry | Horde.Registry
│   ├── Sagents.RegistryWatcher
│   │
│   ├── Sagents.AgentsDynamicSupervisor
│   │   ├── AgentSupervisor ("conversation-1")
│   │   │   ├── AgentServer
│   │   │   └── SubAgentsDynamicSupervisor
│   │   │       ├── SubAgentServer
│   │   │       └── SubAgentServer
│   │   │
│   │   └── AgentSupervisor ("conversation-2")
│   │       └── ...
│   │
│   └── Sagents.FileSystem.FileSystemSupervisor
│       ├── FileSystemServer ({:user, 1})       # Scoped independently
│       ├── FileSystemServer ({:user, 2})
│       └── FileSystemServer ({:project, 42})   # Can be shared across agents
│
└── MyAppWeb.Endpoint                           # after Sagents.Supervisor, on purpose

Why :rest_for_one

An AgentSupervisor and an AgentServer register their :via names once, at start, and nothing re-registers them afterwards. A registry that came back empty underneath them would leave them running but invisible to every lookup — so the next request reads "nothing is running" and starts a second AgentServer for a conversation that already has one, with both persisting state.

:rest_for_one expresses that dependency: a registry failure takes the dynamic supervisors down with it, agents stop, and the next request re-creates them from persisted state. That looks heavy-handed and is the point. Agent state is durable, so a restart is recoverable; a silent duplicate is not.

Sagents.RegistryWatcher is listed immediately after the registry because on both backends the process that owns the registry's ETS tables sits one level below the child Sagents.Supervisor supervises, and both backends restart that process internally with fresh, empty tables. The registry ends up empty while this supervisor never sees a failed child.

Which process that is differs by backend. Under :horde it is Horde.RegistryImpl, which is itself registered as Sagents.Registry. Under :local it is a Registry.Partition, a child of the Registry.Supervisor that holds the name, so watching the name would miss the failure entirely. Sagents.ProcessRegistry.watched_name/0 resolves the difference, and the watcher monitors whatever it returns, stopping when that process dies. That is what puts the restart into the :rest_for_one chain.

A :local registry has a second failure mode, Registry.Supervisor itself dying, which is a failed child and reaches the chain unaided. Watching the partition covers it too, since the partition dies whenever its supervisor does.

Why the Endpoint comes last

OTP shuts children down in reverse start order. Listed after Sagents.Supervisor, the Endpoint stops accepting requests first and the registry is still alive to serve whatever is in flight. Listed before, it keeps serving requests after the registry is gone, and every one of them fails until the BEAM exits.

Correct ordering narrows that window but does not close it, because the node stays reachable for the platform's whole drain period. Wiring Sagents.ready?/0 into a readiness check is what closes it. See Deployments, draining, and readiness.

Flexible Scoping: FileSystemServer lives outside the AgentSupervisor tree, allowing different scoping strategies. For example:

  • User-scoped filesystem: All of a user's conversations share the same files
  • Project-scoped filesystem: Multiple users' conversations on the same project share files
  • Conversation-scoped filesystem: Each conversation has isolated files

Agents reference their filesystem by scope tuple (e.g., filesystem_scope: {:user, 123}), and the FileSystem middleware looks up or starts the appropriate FileSystemServer.

Error Handling

Agent Crashes

If an AgentServer crashes:

  1. Supervisor restarts it
  2. State is lost (unless persisted)
  3. Clients receive {:agent_shutdown, %{reason: :crash}}

To preserve state across crashes, enable auto-save:

AgentServer.start_link(
  agent: agent,
  auto_save: [
    callback: &MyApp.save_state/2,
    interval: 30_000  # Save every 30 seconds
  ]
)

Registry Failure

A registry process crashing is rare, but it has a defined outcome: the :rest_for_one chain restarts the dynamic supervisors below it, agents on that node stop, and the next request re-creates them from persisted state.

Two internal pieces make that restart land correctly, and you call neither:

  • Sagents.RegistryWatcher connects the failure to the restart chain, because both backends restart the process that owns the registry's tables internally, leaving the registry empty without failing a child of Sagents.Supervisor.
  • Sagents.LocalRegistry covers the :local backend, where the restart races the outgoing registry's partition process. That partition traps exits, because it links to every registered process, so it does not die with the registry: it terminates asynchronously and holds its registered name for a few milliseconds longer. A start landing inside that window fails with {:already_started, pid}, and a supervisor does not retry its way out of a failed restart — it would give up and take the whole tree with it. Sagents.LocalRegistry monitors the straggler, waits for it to exit, and retries.

Under :horde this deliberately over-reacts. A surviving peer repairs a restarted registry from the CRDT within a few hundred milliseconds, pointing at the processes that were still running, so on a healthy cluster a registry crash would have cost nothing. The agents are restarted anyway, because no node can tell in time whether that repair is coming: a single-node deployment has no peer, and a partitioned cluster is indistinguishable from a healthy one from the inside. Over-reacting costs a restart from durable state. Under-reacting leaves agents running but unregistered, which is the silent duplicate above.

A registry failure is not part of draining. The watcher sits after the registry and OTP stops children in reverse order, so it is gone before the registry is on an orderly shutdown.

A Draining or Starting Node

The node's registry cannot answer lookups before Sagents.Supervisor has started and after it has shut down. The second window is every rolling deploy, and it lasts for the platform's whole grace period while the node is still reachable.

Lookups report this as {:error, :registry_unavailable} or raise Sagents.RegistryUnavailableError rather than answering with a plausible default — see A lookup has three outcomes. It is not a race to retry through: on the affected node every call fails identically until the BEAM exits, so recovery has to come from the request landing on a different node.

Node Departure in a Cluster (:horde)

Horde hands a departed node's processes to a survivor only once that survivor has converged on the departure and marked the node's member entry dead. An agent placed immediately before its node leaves is dropped rather than handed over; one that has been running for a couple of seconds or more is redistributed reliably. Whether the departure is graceful makes no difference.

A dropped agent is dropped cleanly: the registry and Horde's process CRDT are both left with no entry for it, so there is no orphan and no duplicate, and the next Sagents.Session.ensure_running/3 starts it again from persisted state. That is the same recovery path an inactivity shutdown uses.

The rule to design to: treat redistribution as an optimization and the next request as the guarantee. Work that must happen should be driven by a request, a job, or a supervisor you control, never by assuming Horde kept an agent alive somewhere. See Taking a node out of the cluster.

LLM Errors

LLM API errors are handled gracefully:

case LLMChain.run(chain) do
  {:ok, chain} ->
    # Success
    {:ok, extract_state(chain)}

  {:error, chain, reason} ->
    # Broadcast error, keep state intact
    broadcast(:status_changed, :error, reason)
    {:error, reason}
end

Tool Execution Errors

Tool errors are returned to the LLM as tool results:

# If tool function returns {:error, reason}
%ToolResult{
  tool_call_id: call_id,
  content: "Error: #{reason}",
  is_error: true
}

The LLM can then decide how to proceed (retry, ask user, etc.).

Performance Considerations

Memory

  • Each agent process holds its full message history in memory
  • Use Summarization middleware to compress long conversations
  • FileSystem middleware can offload to persistence callbacks

Concurrency

  • Each agent is independent - no contention between conversations
  • SubAgents run in parallel under the same supervisor
  • Event delivery via Sagents.Publisher is non-blocking send/2 per subscriber and doesn't block execution

Startup Time

  • Agent startup is fast (just GenServer.start_link)
  • State restoration depends on storage backend
  • Consider lazy-loading old messages if history is large