Context compaction (SPEC §7F) as a before_llm helper — no core-loop change.
A long-lived agent grows its transcript until it overflows the model's window.
compactor/1 returns a before_llm hook (§8) that summarizes the older transcript
and keeps a recent tail. It rides the existing seam: the loop applies a before_llm
message rewrite by replacing the working transcript, and that list flows into
RunResult.messages and the ConversationStore — so compaction is a pure
messages → messages helper that adds no loop behavior.
Below max_tokens the hook is a no-op — it returns nil, the loop keeps the
transcript unchanged, and the run is byte-identical to one with no compactor. Above
it, the transcript becomes:
[leading system prompt (verbatim), summary system message, (flush reminder?), …tail]Two invariants hold:
- Tool-pair safety. The retained tail begins at a
userturn, so notoolmessage is ever orphaned from theassistantcarrying itstool_call_id. The split is the largest user-boundary tail that fitskeep_tail; if none fits, it extends to the most recent user turn (safety over size). - System prompt preserved. A leading
systemmessage (identity / soul / skills) is kept unchanged; only the body between it and the tail is summarized.
Messages are plain maps with string keys ("role", "content", "tool_calls",
"tool_call_id"), matching the transcripts Toolnexus.Client builds.
Options (keyword list)
:max_tokens— compact only when the estimate exceeds this; at/below ⇒ no-op. Required.:keep_tail— keep at least this many tokens of the most recent tail (defaultdiv(max_tokens, 2)).:summarize—(older :: [map] -> String.t()); produces the summary. MAY call an LLM — the library makes no model call on the host's behalf by default. Required.:count_tokens—([map] -> non_neg_integer()); token estimate, defaultestimate_tokens/1(ceil(byte_size(json)/4)summed over messages — an estimator, not a tokenizer).:flush_to_memory— when set, inject a pre-compact system reminder to persist durable facts via the §7Ememorytool before the head is summarized (off by default).
Summary
Functions
Build a before_llm hook that compacts the transcript when it grows too large.
Cheap, deterministic token estimate: ceil(byte_size(json)/4) summed over messages.
Functions
Build a before_llm hook that compacts the transcript when it grows too large.
Returns a 1-arity function taking the before_llm event
(%{messages:, tools:, model:, turn:}). It returns %{messages: compacted} only when
it actually compacts; otherwise nil (a no-op the loop leaves untouched).
@spec estimate_tokens([map()]) :: non_neg_integer()
Cheap, deterministic token estimate: ceil(byte_size(json)/4) summed over messages.
This is an estimator, not a tokenizer — override it via the :count_tokens option
when exactness matters.