LangEx.Middleware (LangEx v0.11.3)

Copy Markdown View Source

Composable hooks that wrap the model call inside LangEx.Prebuilt.agent/1.

A middleware is a value — a %LangEx.Middleware{} carrying optional hook functions — so behaviours like summarization, context editing, planning, tool pre-selection, and completion gating compose without each one being hardcoded into the agent. The agent runs every turn through the stack:

before_model (first  last)
   wrap_model_call (first middleware outermost)
     the LLM call
   after_model (last  first)

after_model runs in reverse so the stack unwinds symmetrically: the middleware that saw the state last on the way in sees the result first on the way out.

Hooks

  • :before_model(state -> update) run before the LLM call. Its update is applied to the working state (so the model sees it) and persisted. Return :messages instructions (including LangEx.Message.remove_all/0 / LangEx.Message.remove/1) to rewrite history, not just append.
  • :after_model(state -> update) run after the LLM call. Set :__agent_jump__ to :model (loop again), :tools, or :__end__ in the update to override the agent's routing — e.g. a completion gate that bounces an inadequate answer back for another pass.
  • :wrap_model_call(request, next -> update) wraps the LLM call. The request is %{messages: [...], tools: [...], state: map()}; call next.(request) (optionally with narrowed :tools) to run the model.

Contributions

  • :tools — extra %LangEx.Tool{} the middleware adds to the agent.
  • :state_schema — schema fragment (key: default / key: {default, reducer}) merged into the agent's graph state.

Usage-bearing hooks (a summariser, a critic) should return their token usage under :llm_usage; the runner sums it with the turn's model usage.

Contributed-key semantics

The runner accumulates :messages (concatenating instructions, so remove_all/0/remove/1 work) and :llm_usage (summing) across all hooks. Any other contributed key uses last-write-wins for the turn — a custom reducer declared in :state_schema is still applied once by the graph engine when the turn's update is committed, but is not re-applied between hooks within a turn, so two hooks writing the same custom-reducer key in one turn keep only the last write. Keep middleware state keys last-write-wins (as the built-ins do) to avoid surprise.

Summary

Functions

The reserved state key an after_model hook sets to steer routing.

Builds a middleware from a keyword list of hooks and contributions.

Runs one model turn through the middleware stack.

The merged schema fragment contributed across a middleware stack.

All tools contributed across a middleware stack, in order.

Types

hook()

@type hook() :: (map() -> map())

request()

@type request() :: %{
  messages: [LangEx.Message.t()],
  tools: [LangEx.Tool.t()],
  state: map()
}

t()

@type t() :: %LangEx.Middleware{
  after_model: hook() | nil,
  before_model: hook() | nil,
  name: atom() | nil,
  state_schema: keyword(),
  tools: [LangEx.Tool.t()],
  wrap_model_call: wrapper() | nil
}

wrapper()

@type wrapper() :: (request(), (request() -> map()) -> map())

Functions

jump_key()

@spec jump_key() :: atom()

The reserved state key an after_model hook sets to steer routing.

new(opts)

@spec new(keyword()) :: t()

Builds a middleware from a keyword list of hooks and contributions.

run_turn(state, model_fn, tools, middlewares, messages_key)

@spec run_turn(
  map(),
  (list(), [LangEx.Tool.t()], map() -> map()),
  [LangEx.Tool.t()],
  [t()],
  atom()
) ::
  map()

Runs one model turn through the middleware stack.

model_fn is (messages, tools, state -> update) — the raw LLM call, returning a %{messages_key => [ai], :llm_usage => usage} update. It receives the current working state so state-derived options can be resolved. tools is the full tool list offered to the model (a wrap_model_call hook may narrow it). Returns the merged, persistable update for the agent node.

state_schema(middlewares)

@spec state_schema([t()]) :: keyword()

The merged schema fragment contributed across a middleware stack.

tools(middlewares)

@spec tools([t()]) :: [LangEx.Tool.t()]

All tools contributed across a middleware stack, in order.