Normandy.Context.WindowManager (normandy v2.2.0)

View Source

Manages context window limits for conversations with automatic truncation strategies.

Provides utilities for tracking token usage, managing context window limits, and automatically truncating conversation history when approaching limits.

Features

  • Token counting for messages and conversations
  • Context window passed in by the caller (no built-in model table)
  • Multiple truncation strategies (oldest-first, sliding window, summarization)
  • Automatic context management with AgentMemory integration

Example

# Create a window manager with the model's context window
manager = WindowManager.new(max_tokens: 200_000)

# Check if we're within limits
{:ok, within_limit?} = WindowManager.within_limit?(manager, agent)

# Automatically truncate if needed
case WindowManager.ensure_within_limit(agent, manager) do
  {:ok, updated_agent} -> updated_agent
  {:error, reason} -> handle_summary_failure(agent, reason)
end

Truncation Strategies

  • :oldest_first - Remove oldest messages first (default)
  • :summarize - Summarize old messages before removing; needs the agent's :client (a %BaseAgentConfig{} always has one) and raises ArgumentError without it

The kept messages always start at a user message, so the conversation stays valid for the API (no leading assistant turn, no tool result without its tool use). If the newest messages alone exceed the limit, their whole turn is kept rather than an empty history.

Summary

Types

strategy()
t()

Functions

ensure_within_limit(agent, manager)

Ensures agent's conversation stays within context window limit.

estimate_conversation_tokens(memory)

Estimates total tokens in conversation history.

estimate_message_content_tokens(content)

Estimates token count for message content.

estimate_tokens(text)

Estimates token count for a message.

new(opts \\ [])

Creates a new WindowManager.

truncate_conversation(agent, manager)

Truncates conversation history using the configured strategy.

within_limit?(manager, agent)

Checks if current conversation is within token limit.

Types

strategy()

@type strategy() :: :oldest_first | :summarize

t()

@type t() :: %Normandy.Context.WindowManager{
  max_tokens: pos_integer(),
  reserved_tokens: non_neg_integer(),
  strategy: strategy(),
  summary_max_tokens: pos_integer() | nil
}

Functions

ensure_within_limit(agent, manager)

@spec ensure_within_limit(
  struct(),
  t()
) :: {:ok, struct()} | {:error, term()}

Ensures agent's conversation stays within context window limit.

Automatically truncates history if needed using the configured strategy. Returns {:error, reason} when the :summarize strategy's summarization fails, leaving the agent unchanged.

Example

{:ok, updated_agent} = WindowManager.ensure_within_limit(agent, manager)

estimate_conversation_tokens(memory)

@spec estimate_conversation_tokens(Normandy.Components.AgentMemory.t()) ::
  non_neg_integer()

Estimates total tokens in conversation history.

Example

tokens = WindowManager.estimate_conversation_tokens(agent.memory)

estimate_message_content_tokens(content)

@spec estimate_message_content_tokens(term()) :: non_neg_integer()

Estimates token count for message content.

Example

tokens = WindowManager.estimate_message_content_tokens("Hello")
#=> ~2

estimate_tokens(text)

@spec estimate_tokens(String.t()) :: non_neg_integer()

Estimates token count for a message.

This is a rough estimate based on character count. For accurate counts, use Normandy.Context.TokenCounter.count_message/3.

Example

tokens = WindowManager.estimate_tokens("Hello, world!")
#=> ~4 tokens

new(opts \\ [])

@spec new(keyword() | map()) :: t()

Creates a new WindowManager.

Options

  • :max_tokens (pos_integer/0) - Required. The model's context window in tokens. Inside an agent the compactor passes the window resolved through the configured Normandy.Behaviours.ModelCatalog; standalone, take it from Claudio.Models.get/2 ("max_input_tokens").

  • :reserved_tokens (non_neg_integer/0) - Tokens kept free for the response. The default value is 4096.

  • :strategy - The truncation strategy. The default value is :oldest_first.

  • :summary_max_tokens - max_tokens of the :summarize strategy's summary call; nil is Normandy.Context.Summarizer's default (the agent's max_tokens, else 16_000). The default value is nil.

Example

manager = WindowManager.new(max_tokens: 200_000, strategy: :oldest_first)

truncate_conversation(agent, manager)

@spec truncate_conversation(
  struct(),
  t()
) :: {:ok, struct()} | {:error, term()}

Truncates conversation history using the configured strategy.

Returns {:error, reason} when the :summarize strategy's summarization fails, leaving the agent unchanged.

Example

case WindowManager.truncate_conversation(agent, manager) do
  {:ok, updated_agent} -> updated_agent
  {:error, reason} -> handle_summary_failure(agent, reason)
end

within_limit?(manager, agent)

@spec within_limit?(
  t(),
  struct()
) :: {:ok, boolean()}

Checks if current conversation is within token limit.

Example

case WindowManager.within_limit?(manager, agent) do
  {:ok, true} -> :continue
  {:ok, false} -> :truncate_needed
end