View Source Agens.Router behaviour (agens v0.2.0)
Behaviour and macro for routing a Node's result to the next instruction(s).
A Router maps the outputs produced by a Serving onto a list of route instructions
(e.g. {:route, node_id, 1}, {:yield, node_id}, :end, :retry). The Router can
be implemented either:
- Merged - in the Serving module itself by
use Agens.Routeralongsideuse Agens.Serving. - Split - in a dedicated module reused across Servings, passed via
use Agens.Serving, router: MyRouter.
Implementing a Router
A Router must implement two callbacks:
outputs/1- declares theAgens.Router.Outputschema for the Node's structured outputs.resolve/2- maps a populated outputs list to a list of route instructions.
Using use Agens.Router injects a route/1 and route/2 function that combine these:
route/1 validates the Serving's structured outputs against the declared schema and calls
resolve/2. route/2 accepts a dynamic LM-supplied next list and merges it via parse_next/1,
falling back to route/1 when the dynamic list resolves to no usable instructions.
LM-driven dynamic routing
When a Serving's structured response includes a non-empty next list, those entries are decoded
by parse_next/1 and used in place of the static Router's resolution.
Summary
Callbacks
Declares the Agens.Router.Output schema for a Node's structured outputs.
Maps the populated Agens.Router.Output list onto a list of route instructions.
Functions
Decodes an LM-supplied next list (a list of %{"type" => ..., "value" => ...} maps)
into Agens route instructions.
Callbacks
@callback outputs(Agens.Message.t()) :: [Agens.Router.Output.t()]
Declares the Agens.Router.Output schema for a Node's structured outputs.
The returned list is used both to validate the Serving's outputs (every declared key must
be present) and as the input to resolve/2.
@callback resolve(Agens.Message.t(), [Agens.Router.Output.t()]) :: list()
Maps the populated Agens.Router.Output list onto a list of route instructions.
Functions
Decodes an LM-supplied next list (a list of %{"type" => ..., "value" => ...} maps)
into Agens route instructions.
Unknown entries are dropped. Supported types: "route", "yield", "sub", "end", "retry".