Toolnexus.Agents.Compaction (toolnexus v0.12.0)

Copy Markdown View Source

Context compaction (SPEC §7F) as a before_llm helper — no core-loop change.

A long-lived agent grows its transcript until it overflows the model's window. compactor/1 returns a before_llm hook (§8) that summarizes the older transcript and keeps a recent tail. It rides the existing seam: the loop applies a before_llm message rewrite by replacing the working transcript, and that list flows into RunResult.messages and the ConversationStore — so compaction is a pure messages → messages helper that adds no loop behavior.

Below max_tokens the hook is a no-op — it returns nil, the loop keeps the transcript unchanged, and the run is byte-identical to one with no compactor. Above it, the transcript becomes:

[leading system prompt (verbatim), summary system message, (flush reminder?), tail]

Two invariants hold:

  • Tool-pair safety. The retained tail begins at a user turn, so no tool message is ever orphaned from the assistant carrying its tool_call_id. The split is the largest user-boundary tail that fits keep_tail; if none fits, it extends to the most recent user turn (safety over size).
  • System prompt preserved. A leading system message (identity / soul / skills) is kept unchanged; only the body between it and the tail is summarized.

Messages are plain maps with string keys ("role", "content", "tool_calls", "tool_call_id"), matching the transcripts Toolnexus.Client builds.

Options (keyword list)

  • :max_tokens — compact only when the estimate exceeds this; at/below ⇒ no-op. Required.
  • :keep_tail — keep at least this many tokens of the most recent tail (default div(max_tokens, 2)).
  • :summarize(older :: [map] -> String.t()); produces the summary. MAY call an LLM — the library makes no model call on the host's behalf by default. Required.
  • :count_tokens([map] -> non_neg_integer()); token estimate, default estimate_tokens/1 (ceil(byte_size(json)/4) summed over messages — an estimator, not a tokenizer).
  • :flush_to_memory — when set, inject a pre-compact system reminder to persist durable facts via the §7E memory tool before the head is summarized (off by default).

Summary

Functions

Build a before_llm hook that compacts the transcript when it grows too large.

Cheap, deterministic token estimate: ceil(byte_size(json)/4) summed over messages.

Functions

compactor(opts)

@spec compactor(keyword()) :: (map() -> %{messages: [map()]} | nil)

Build a before_llm hook that compacts the transcript when it grows too large.

Returns a 1-arity function taking the before_llm event (%{messages:, tools:, model:, turn:}). It returns %{messages: compacted} only when it actually compacts; otherwise nil (a no-op the loop leaves untouched).

estimate_tokens(messages)

@spec estimate_tokens([map()]) :: non_neg_integer()

Cheap, deterministic token estimate: ceil(byte_size(json)/4) summed over messages.

This is an estimator, not a tokenizer — override it via the :count_tokens option when exactness matters.