Behaviour for context-window compaction.
When the conversation's estimated token footprint crosses
:compact_at (fraction of the provider's max_tokens), the loop asks
the Compactor to reduce history size. The Compactor's job is to preserve
intent + pinned rules while replacing the middle of history with
a summary.
Contract
- Pinned prefix: the first N messages (
:pinned_prefix_count) are never dropped. System prompts + CLAUDE.md-style pinned rules live there. - Live suffix: the last K messages (
:live_suffix_count) are never dropped. Recent context the model needs to keep reasoning. - Middle: everything between is the Compactor's to replace. It may emit zero or more summary messages that sit where the dropped messages used to be.
Default implementation ExAthena.Compactors.Summary uses the session's
own provider to generate a terse summary message and substitutes it.
Consumers can swap in any module via config :ex_athena, compactor: MyApp.MyCompactor.
Why
Research (Anthropic compact_20260112 beta, Cline, Claude Agent SDK):
proactive compaction at ~60% of the context limit beats reactive
truncation at 95% — the model never notices a sudden loss of
continuity, and pinned rules survive every compaction cycle.
Summary
Callbacks
Run compaction against the current state. Return one of
Whether compaction should run this turn. The kernel calls this before
compact/2 so the compactor can defer cheaply without having to build
a summary.
Functions
Best-effort token estimator. Counts ~4 chars per token for text content, plus a small fixed cost per tool-call to cover the JSON envelope. Good enough for compaction triggers; not a billing number.
Recompute an estimate's :tokens for a new message list, keeping the
system-prompt cost included.
The system prompt the loop sends with every request, read from the
state's request template. nil when the state carries none.
Types
@type decision() :: {:compact, messages :: [ExAthena.Messages.Message.t()], metadata :: map()} | :skip | {:error, term()}
@type estimate() :: %{tokens: non_neg_integer(), max_tokens: non_neg_integer()}
Callbacks
@callback compact(ExAthena.Loop.State.t(), estimate()) :: decision()
Run compaction against the current state. Return one of:
{:compact, new_messages, metadata}— the kernel swapsstate.messagesfornew_messagesand emits a{:compaction, …}event withmetadata.:skip— do nothing this cycle (e.g. compactor judged compaction not yet necessary). The kernel emits no event.{:error, reason}— terminate the run with:error_compaction_failed.
@callback should_compact?(ExAthena.Loop.State.t(), estimate()) :: boolean()
Whether compaction should run this turn. The kernel calls this before
compact/2 so the compactor can defer cheaply without having to build
a summary.
Functions
@spec estimate_tokens([ExAthena.Messages.Message.t()], String.t() | nil) :: non_neg_integer()
Best-effort token estimator. Counts ~4 chars per token for text content, plus a small fixed cost per tool-call to cover the JSON envelope. Good enough for compaction triggers; not a billing number.
Pass the request's system_prompt as the second argument to include its
cost in the estimate. The system prompt (which carries tool descriptions
and skill catalogs) is part of every request and can run to thousands of
tokens, so omitting it makes the proactive should_compact? trigger fire
later than it should — or not at all on prompt-heavy agents.
@spec re_estimate( estimate(), [ExAthena.Messages.Message.t()], ExAthena.Loop.State.t() ) :: estimate()
Recompute an estimate's :tokens for a new message list, keeping the
system-prompt cost included.
Pipeline stages MUST use this (not estimate_tokens/1) when they
re-estimate after shrinking the conversation: the system prompt rides on
every request, so an estimate recomputed from messages alone lets the
pipeline declare itself under target while the real request is still
oversized — the same failure mode that got ContextCollapse removed
from the default pipeline.
@spec system_prompt(ExAthena.Loop.State.t()) :: String.t() | nil
The system prompt the loop sends with every request, read from the
state's request template. nil when the state carries none.