An append-only event log with a model-visible surface derived by folding.
The log is the source of truth. derive_messages/1 projects it into the
[%Nous.Message{}] list every existing caller already reads, so the log can be
internal while result.messages stays byte-identical (plan constraint D2).
Replace, don't delete
{:ok, log} = Log.append(log, :assistant_message, %{content: "…"})
{:ok, log} = Log.append(log, :system_message, %{
content: "[summary of 1-40]",
surface_op: {:replace, 1, 40}
})After that append, derive_messages/1 shows the summary in place of events 1
through 40, while events/1 still returns all of them. Compaction stops
destroying history — which is the whole point, and what makes fork, rewind and
audit possible later.
Cost
Folding on every read would make the O(n) append this replaced look cheap. The
fold is memoized against replace_generation — a counter bumped only by a
replace — plus the event count, so an ordinary append extends the cached list
instead of rebuilding it, and a replace invalidates it exactly once.
Bounding the log: snapshot/1
A replace shadows events; it does not remove them, so a long-running session
that is summarised again and again keeps every original event plus every
superseded summary, and its persisted blob grows with the number of
compactions. snapshot/1 drops every shadowed event and keeps the rest with
their seqs intact, so the log becomes O(live history). It is a deliberate,
explicit trade: after a snapshot the shadowed history is gone from this log,
so fork/rewind past that point is no longer possible. Nothing calls it
implicitly; Nous.Plugins.Summarization does so only with
snapshot_log: true.
What is not in the log
The system prompt rewrite performed per request by
Nous.AgentRunner.PromptAssembly is assembly-time state, not history: it is
derived from agent config, plugins and skills at request time, it is replaced
wholesale on every request, and logging each rewrite would append N copies of
the same text to a durable log. System messages that are genuinely part of the
transcript — the initial one, and the summary
Nous.Plugins.Summarization appends — are :system_message events like any
other. See Nous.Session.Event.
Summary
Functions
Append an event.
Append an event, logging and returning the log unchanged on failure.
Restore an event with the seq it was persisted under.
The number of events ever appended — shadowed ones, and ones a snapshot/1
has dropped, included. Equivalently, the next seq.
Fold the surface into {seq, message} pairs, in the order the model sees them.
The message list every caller sees: derive_indexed/1 without the seqs.
Every event ever appended, oldest first, including shadowed ones.
derive_messages/1, returning the log with the fold memoized.
An empty log.
Seed a log from a flat message list, one surface event per message, in order.
Events from seq onward, oldest first. Costs O(number returned), not O(log).
Drop every shadowed event, keeping the survivors' seqs.
The surface events, in the order the model sees them: surface-typed events that no later replace has shadowed.
Types
@type t() :: %Nous.Session.Log{ cache: {non_neg_integer(), non_neg_integer(), [Nous.Message.t()]} | nil, events: [Nous.Session.Event.t()], next_seq: non_neg_integer(), replace_generation: non_neg_integer() }
events is kept newest-first so an append is O(1); events/1 reverses.
cache holds {replace_generation, event_count, messages} — see the moduledoc
on cost.
Functions
@spec append(t(), Nous.Session.Event.type(), map(), DateTime.t() | nil) :: {:ok, t()} | {:error, term()}
Append an event.
seq is assigned here and is always contiguous — it equals the event's index,
which is what lets {:replace, start, stop} name a range without storing
pointers.
Returns {:error, reason} on an invalid event; a bad event must never take down
a live run.
time defaults to now. Pass it explicitly when the event records something that
already happened, which is required in two places: appending an existing
%Nous.Message{} must reproduce that message's own created_at (the fold
stamps from the event time, so re-stamping would make add_message/2 lossy),
and rebuilding a log from persisted events must keep their original times rather
than the moment of the restore.
Examples
iex> log = Nous.Session.Log.new()
iex> {:ok, log} = Nous.Session.Log.append(log, :user_message, %{content: "hi"})
iex> {:ok, log} = Nous.Session.Log.append(log, :assistant_message, %{content: "hello"})
iex> Enum.map(Nous.Session.Log.events(log), & &1.seq)
[0, 1]@spec append!(t(), Nous.Session.Event.type(), map(), DateTime.t() | nil) :: t()
Append an event, logging and returning the log unchanged on failure.
For call sites inside the agent loop, where losing one bookkeeping event is strictly better than failing the user's run. The failure is never silent.
@spec append_at( t(), non_neg_integer(), Nous.Session.Event.type(), map(), DateTime.t() | nil ) :: {:ok, t()} | {:error, term()}
Restore an event with the seq it was persisted under.
append/4 numbers events itself; this is for rebuilding a log from persisted
events, where the seqs must survive verbatim — a {:replace, start, stop}
names positions, and after a snapshot/1 the persisted seqs legitimately
have gaps. seq must be at or past the next seq: restoring behind the head
would either duplicate a seq or reorder history, and is rejected.
@spec count(t()) :: non_neg_integer()
The number of events ever appended — shadowed ones, and ones a snapshot/1
has dropped, included. Equivalently, the next seq.
@spec derive_indexed(t()) :: [{non_neg_integer(), Nous.Message.t()}]
Fold the surface into {seq, message} pairs, in the order the model sees them.
This is the primitive; derive_messages/1 drops the seqs. Compaction needs the
pairs: to replace "these messages" it has to name a seq range, and the mapping
from message position to originating event is exactly what the projection rules
decide. Re-deriving that mapping outside this module means re-implementing the
drop rule below, which is how the two would silently diverge.
Projection rules:
:system_message,:user_message,:assistant_message→ a message of that role, reproduced field for field:tool_result→ a:toolmessage carryingtool_call_idandname- everything else → nothing
Nothing else is filtered. An assistant event with neither content nor tool calls
still projects: it is what happened, and dropping it made
Nous.Agent.Context.last_message/1 and output extraction disagree with the log.
Memoized; see the moduledoc on cost.
@spec derive_messages(t()) :: [Nous.Message.t()]
The message list every caller sees: derive_indexed/1 without the seqs.
@spec events(t()) :: [Nous.Session.Event.t()]
Every event ever appended, oldest first, including shadowed ones.
@spec materialize(t()) :: {[Nous.Message.t()], t()}
derive_messages/1, returning the log with the fold memoized.
Callers that hold the log (Nous.Agent.Context) should use this so a later read
is free. The cache holds the indexed pairs, so a later derive_indexed/1 is
free too.
@spec new() :: t()
An empty log.
@spec seed([Nous.Message.t()]) :: t()
Seed a log from a flat message list, one surface event per message, in order.
This is the v1→v2 persistence bridge and what Nous.run(agent, messages: [...])
uses. Messages with an unrecognised role are dropped with a warning rather than
guessed at.
Each event takes its message's own created_at as its time, so seeding is
lossless: seed/1 |> derive_messages/1 returns the input list, timestamps
included. Re-stamping here would make Nous.Agent.Context.new(messages: …)
quietly rewrite the history it was handed.
@spec since(t(), non_neg_integer()) :: [Nous.Session.Event.t()]
Events from seq onward, oldest first. Costs O(number returned), not O(log).
events/1 reverses the whole list, which is fine for a one-off read and wrong
for anything on the append path — publishing newly committed events runs on
every append, and so does a consumer catching up from a known seq. Because
events are stored newest-first, walking while event.seq >= seq touches only
what is new — and, unlike counting, stays correct after a snapshot/1 has
left gaps in the seq sequence.
seq at or beyond the end returns [].
Examples
iex> log = Nous.Session.Log.new()
iex> {:ok, log} = Nous.Session.Log.append(log, :user_message, %{content: "a"})
iex> {:ok, log} = Nous.Session.Log.append(log, :user_message, %{content: "b"})
iex> Nous.Session.Log.since(log, 1) |> Enum.map(& &1.data.content)
["b"]
iex> Nous.Session.Log.since(log, 5)
[]Drop every shadowed event, keeping the survivors' seqs.
The surface (surface/1, derive_messages/1) is unchanged — a shadowed event
contributes nothing to it by definition — so the memoized fold survives too.
count/1 keeps counting from the original head: seqs are never reused.
Examples
iex> log = Nous.Session.Log.new()
iex> {:ok, log} = Nous.Session.Log.append(log, :user_message, %{content: "one"})
iex> {:ok, log} = Nous.Session.Log.append(log, :user_message, %{content: "two"})
iex> {:ok, log} = Nous.Session.Log.append(log, :system_message, %{content: "[summary]", surface_op: {:replace, 0, 1}})
iex> log = Nous.Session.Log.snapshot(log)
iex> Enum.map(Nous.Session.Log.events(log), & &1.seq)
[2]
iex> Nous.Session.Log.count(log)
3@spec surface(t()) :: [Nous.Session.Event.t()]
The surface events, in the order the model sees them: surface-typed events that no later replace has shadowed.
A replacing event takes the position of the range it replaces, not its own
append position. A summary of events 0–40 belongs where those events were; put
at the end it would read as "here is the conversation, and now a summary of the
part that came before it", and it would leave a :tool message at the head with
no assistant prelude, which every provider rejects.
Examples
iex> log = Nous.Session.Log.new()
iex> {:ok, log} = Nous.Session.Log.append(log, :user_message, %{content: "one"})
iex> {:ok, log} = Nous.Session.Log.append(log, :step_start, %{})
iex> {:ok, log} = Nous.Session.Log.append(log, :assistant_message, %{content: "two"})
iex> Enum.map(Nous.Session.Log.surface(log), & &1.type)
[:user_message, :assistant_message]