Memory model and sizing

This is the through-line that ties the agent, persistence, compaction, and rendering together. The core rule: the agent holds the bounded working set; the log holds everything; the renderer reads the log for scrollback. Conflating these is what would quietly turn every agent into a memory hog.

Three stores, two of them not in the agent

• The working set (in memory, bounded). What the agent keeps: the small fsm_state, the rendered context for the active turn (already bounded — it's what compaction produces: latest summary + verbatim tail + retained/stubbed tool results, under budget), a short tail of recent finalized messages so it can assemble the next turn without a DB round-trip, and the in-progress assistant text it accumulates mid-stream (it finalizes the message from this buffer and serves it to mid-stream mounts — see 06). The agent never holds message 1 through 4,000 in memory.
• The log (on disk, unbounded). The events table is the only unbounded store and the source of truth: every message, the real tool results (not the stubs), every suspension and resolution. The agent reads from it lazily — on revival it reconstructs the working set from "latest summary + events since that summary's span," not by loading everything.
• Scrollback (renderer, reads the log). When a user scrolls up to a message long since compacted out of the working set, the headless layer paginates events backward. The UI shows the whole conversation; the agent only ever holds the working set.

There is exactly one unbounded store and one bounded store. No third copy, no "agent's idea of history" drifting from "the log's idea of history." The rendered context the agent keeps and the thing it sends the model are the same bounded object, a projection of the log.

Consequence: agent footprint is flat in conversation length and linear in the active agent count (managed by eviction), not in total history.

User sees the log; the model sees the projection

A deliberate, correct divergence: scrolling up shows the true historical message from the log; the model gets the compacted projection. They answer different questions ("what was said" vs "what fits the budget"). The practical upshot is a debugging rule: when chasing "why did it forget X," pull from model_calls (what was actually rendered to the model that turn), not from what's on screen. That's what the optional audit table is for.

Sizing: bound by tokens, not message count

"Keep N messages" is the wrong knob — one 50 KB tool result is worth ~200 chat messages. The working set is bounded by a token budget, and message count floats under it.

Conversion at ~4 bytes/token and a mixed average of ~200 tokens/message:

Default planning number: ~30K-token working window ≈ 100–150 chat messages. Tool-heavy conversations hold fewer distinct events but more tokens each — except the retention reducer stubs old tool results down to ~10 tokens, so many stubbed pairs cost almost nothing and the count can run higher.

When compaction triggers

Two triggers; for a tool-centric agent the cheap one dominates:

• The deterministic reducers (tool-result retention, sliding window) run on every assembly — they're free, so they're never gated.
• Expensive summarization fires only when the verbatim tail exceeds the working budget — a cost knob set well below the model ceiling, since every context token is re-billed every turn. Default working budget ~25–50K tokens; hard ceiling ~70–75% of the model window as the never-exceed guard that approximate token-counting's safety margin protects.

Memory per gen_statem

Text content lives in refc binaries: anything over 64 bytes (essentially every message) goes to the shared binary heap as a ref-counted binary, so the process heap holds a ~56-byte pointer, not the bytes. The dominant cost is binary, and if the working set is kept as a ReqLLM.Context of message structs (not a pre-concatenated prompt string), the same binaries are shared between the message list and the assembled context rather than duplicated.

For a ~32K-token chat working set (~160 messages):

Typical 150–250 KB for an active chat agent. Between GC sweeps RSS runs ~1.5–2×, so budget ~300–400 KB. Tool-heavy with a few large recent results still in full (say 3 × 6K tokens before they age out) adds ~70 KB plus cheap stubs → ~300–600 KB.

Scaling sanity check (text isn't shared across conversations, so it's linear in the active set):

Which is exactly why eviction to the log earns its keep, and why footprint must stay flat in length.

Constraints this imposes (honor from day one)

• Keep the working set as ReqLLM.Context, never a concatenated prompt string. Concatenating doubles the footprint and breaks binary sharing.
• Keep the in-memory message struct lean. Per-turn usage, latency, and audit go to model_calls on disk, not into the in-memory struct — otherwise every process carries data only the debugger wants.
• Hibernate before evicting. See 01 — hibernate compacts the heap for agents parked in awaiting_input; full idle terminates and persists to the log.