GenAgent.Backend behaviour (GenAgent v0.6.2)

Behaviour for GenAgent LLM backends.

A backend is responsible for:

Starting and terminating a persistent session with an LLM.
Translating a prompt into a stream of GenAgent.Event values.
Optionally tracking state that changes between turns (e.g. a session id assigned by the backend after the first response).

The state machine owns backend lifecycle. It calls start_session/1 once when the agent boots, prompt/2 (or optional prompt/3) on each turn, and terminate_session/1 from its termination callback. These three callbacks are required; prompt/3, checkpoint_session/2, update_session/2, and resume_session/2 are optional.

Session values

A session is an opaque term owned by the backend. It is passed back to the backend on each call so that the backend can carry private state across turns without needing its own process. Backends that need a process of their own can store its pid inside the session term.

Because prompt/2 can mutate session state (for example, capturing a session_id the first time the backend sees one), it returns an updated session alongside the event stream. The state machine replaces its cached session with whatever prompt/2 returns.

Event stream contract

The enumerable returned from prompt/2 must:

Yield GenAgent.Event values in arrival order.
Emit exactly one terminal event (:result or :error) as the final element. The state machine stops consuming the stream after a terminal event.
Be safe to consume from inside a Task (the state machine runs prompt execution under a Task.Supervisor).

Streams may be lazy. A lazy stream that blocks until fresh events arrive is the expected shape for backends that wrap long-running CLIs.

Cancellation and cleanup

GenAgent stops the active prompt task on interruption, watchdog timeout, or agent exit, including abrupt exits that bypass termination callbacks. Backends must not rely on terminate_session/1 or a stream finalizer running after such an exit.

Stopping a BEAM task does not establish that a provider's subprocess or remote request has stopped. The backend and its transport are responsible for external cancellation and resource cleanup.

Summary

Types

session()

Opaque session term owned by the backend.

Callbacks

checkpoint_session(session, session_id)

Apply an acknowledged CLI identifier to a backend session.

prompt(session, prompt)

Dispatch a prompt on a session.

prompt(session, prompt, context)

Dispatch a prompt with a request-scoped session checkpoint.

resume_session(session_id, opts)

Resume a previously-persisted session by id.

start_session(opts)

Start a new session with the backend.

terminate_session(session)

Tear down a session.

update_session(session, event_data)

Fold a terminal event's data into the session.

Types

session()

@type session() :: term()

Opaque session term owned by the backend.

Callbacks

checkpoint_session(session, session_id)

(optional)

@callback checkpoint_session(session(), session_id :: String.t()) :: session()

Apply an acknowledged CLI identifier to a backend session.

prompt(session, prompt)

@callback prompt(session(), prompt :: String.t()) ::
  {:ok, Enumerable.t(GenAgent.Event.t()), session()} | {:error, term()}

Dispatch a prompt on a session.

Returns an event stream and an updated session. The state machine replaces its cached session with the returned value before the next call.

A synchronous {:error, reason} return indicates the prompt could not be dispatched at all (e.g. the backend process is down). Errors that occur mid-turn should be delivered via an :error terminal event on the stream instead.

prompt(session, prompt, context)

(optional)

@callback prompt(
  session(),
  prompt :: String.t(),
  context :: %{checkpoint: (term() -> :ok | {:error, atom()})}
) :: {:ok, Enumerable.t(GenAgent.Event.t()), session()} | {:error, term()}

Dispatch a prompt with a request-scoped session checkpoint.

context.checkpoint.(id) synchronously acknowledges a nonempty CLI session identifier while this prompt task is active. The first identifier wins; repeated identical identifiers are accepted. Invalid, conflicting, stale, or caller-mismatched checkpoints return {:error, reason}. A backend must fail its turn when a present identifier is rejected. The checkpoint is independent of the normalized event stream and its capture limits.

Optional. Backends without this callback continue to use prompt/2.

resume_session(session_id, opts)

(optional)

@callback resume_session(session_id :: String.t(), opts :: keyword()) ::
  {:ok, session()} | {:error, term()}

Resume a previously-persisted session by id.

Optional. Reserved for reattaching to a session across restarts. GenAgent does not currently call this callback or persist sessions.

start_session(opts)

@callback start_session(opts :: keyword()) :: {:ok, session()} | {:error, term()}

Start a new session with the backend.

Called once when the agent boots. opts are the backend options returned from the implementation's GenAgent.init_agent/1.

terminate_session(session)

@callback terminate_session(session()) :: :ok

Tear down a session.

Called from the agent's termination callback on shutdown and crashes that run that callback. Abrupt exits such as :kill bypass it. Backends should release any resources held by the session here and make cleanup idempotent.

update_session(session, event_data)

(optional)

@callback update_session(session(), event_data :: map()) :: session()

Fold a terminal event's data into the session.

Called once per turn, when the terminal :result event arrives, with the terminal event's :data map. Backends use this hook to capture identifiers assigned after the first response (for example, Claude's session_id).

Optional. If not implemented, the session is unchanged.