Normandy.Context.Summarizer (normandy v0.6.0)

View Source

Handles conversation summarization for context window management.

When conversations exceed token limits, this module can summarize older messages to preserve context while reducing token usage.

Example

# Summarize old messages
{:ok, summary} = Summarizer.summarize_messages(client, agent, messages)

# Replace old messages with summary
{:ok, updated_agent} = Summarizer.compress_conversation(client, agent, keep_recent: 5)

Summary

Functions

compress_conversation(client, agent, opts \\ [])

Compresses a conversation by summarizing old messages.

estimate_savings(messages, opts \\ [])

Estimates token savings from summarization.

summarize_messages(client, agent, messages, opts \\ [])

Summarizes a list of messages using the LLM.

Functions

compress_conversation(client, agent, opts \\ [])

@spec compress_conversation(struct(), struct(), keyword()) ::
  {:ok, struct()} | {:error, term()}

Compresses a conversation by summarizing old messages.

Keeps recent messages intact and replaces older messages with a summary.

Options

  • :keep_recent - Number of recent messages to keep (default: 10)
  • :summary_role - Role for summary message (default: "system")
  • :max_summary_tokens - Max tokens for summary (default: 500)

Example

{:ok, updated_agent} = Summarizer.compress_conversation(
  client,
  agent,
  keep_recent: 5
)

estimate_savings(messages, opts \\ [])

@spec estimate_savings(
  list(),
  keyword()
) :: {:ok, map()}

Estimates token savings from summarization.

Example

{:ok, savings} = Summarizer.estimate_savings(messages, summary_tokens: 200)
#=> %{original: 1500, summary: 200, savings: 1300, savings_percent: 86.7}

summarize_messages(client, agent, messages, opts \\ [])

@spec summarize_messages(struct(), struct(), list(), keyword()) ::
  {:ok, String.t()} | {:error, term()}

Summarizes a list of messages using the LLM.

Options

  • :prompt - Custom summarization prompt (default: built-in prompt)
  • :model - Model to use for summarization (default: from agent config)
  • :max_tokens - Maximum tokens for summary (default: 500)

Example

messages = [
  %{role: "user", content: "Hello"},
  %{role: "assistant", content: "Hi there!"}
]

{:ok, summary} = Summarizer.summarize_messages(client, agent, messages)
#=> "User greeted, assistant responded"