View Source Agens.Router behaviour (agens v0.2.0)

Behaviour and macro for routing a Node's result to the next instruction(s).

A Router maps the outputs produced by a Serving onto a list of route instructions (e.g. {:route, node_id, 1}, {:yield, node_id}, :end, :retry). The Router can be implemented either:

  • Merged - in the Serving module itself by use Agens.Router alongside use Agens.Serving.
  • Split - in a dedicated module reused across Servings, passed via use Agens.Serving, router: MyRouter.

Implementing a Router

A Router must implement two callbacks:

Using use Agens.Router injects a route/1 and route/2 function that combine these: route/1 validates the Serving's structured outputs against the declared schema and calls resolve/2. route/2 accepts a dynamic LM-supplied next list and merges it via parse_next/1, falling back to route/1 when the dynamic list resolves to no usable instructions.

LM-driven dynamic routing

When a Serving's structured response includes a non-empty next list, those entries are decoded by parse_next/1 and used in place of the static Router's resolution.

Summary

Callbacks

Declares the Agens.Router.Output schema for a Node's structured outputs.

Maps the populated Agens.Router.Output list onto a list of route instructions.

Functions

Decodes an LM-supplied next list (a list of %{"type" => ..., "value" => ...} maps) into Agens route instructions.

Callbacks

@callback outputs(Agens.Message.t()) :: [Agens.Router.Output.t()]

Declares the Agens.Router.Output schema for a Node's structured outputs.

The returned list is used both to validate the Serving's outputs (every declared key must be present) and as the input to resolve/2.

@callback resolve(Agens.Message.t(), [Agens.Router.Output.t()]) :: list()

Maps the populated Agens.Router.Output list onto a list of route instructions.

Functions

@spec parse_next([map()]) :: list()

Decodes an LM-supplied next list (a list of %{"type" => ..., "value" => ...} maps) into Agens route instructions.

Unknown entries are dropped. Supported types: "route", "yield", "sub", "end", "retry".