LangEx.Middleware.ContextEditing (LangEx v0.11.3)

Copy Markdown View Source

Middleware that clears the contents of stale tool results.

Distinct from summarisation: instead of dropping or condensing rounds, it keeps the full message skeleton (every tool call and reply stays in place) but blanks the body of large, older %Message.Tool{} results — the outputs the model has already reasoned over and no longer needs verbatim. The rewrite persists via LangEx.Message.remove_all/0, and is idempotent: once a result is replaced by the short placeholder it falls under the threshold and is left alone.

Edits are batched behind :trigger_at_chars: rewriting history invalidates the provider's prompt-cache prefix (a cache write costs ~12x a cache read on Anthropic), so clearing a little every turn pays that invalidation repeatedly. Instead the conversation is left untouched until its total content size crosses the trigger, then every eligible result is cleared in one pass — rare invalidations, large reclaim.

No LLM call — cheap, deterministic context recovery. Composes with LangEx.Middleware.Summarization; typically you reach for one or the other.

Options

  • :keep_last - most recent tool results left untouched (default 3)
  • :clear_at_chars - only results larger than this are cleared (default 4_000)
  • :trigger_at_chars - no editing until the total content size of the conversation exceeds this (default 100_000; 0 edits every turn)
  • :placeholder - replacement content for cleared results

Summary

Functions

Builds a context-editing middleware. See the module doc for options.

Functions

new(opts \\ [])

@spec new(keyword()) :: LangEx.Middleware.t()

Builds a context-editing middleware. See the module doc for options.