LemonAgent.Context (lemon_agent v0.1.0)

View Source

Context management utilities for agent conversations.

This module provides functions for managing conversation context size, including estimation, truncation, and warnings. Context management is critical for:

  • Memory efficiency: Preventing unbounded message accumulation
  • Token budget: Staying within model context window limits
  • Cost control: Larger contexts cost more tokens/money
  • Performance: Smaller contexts stream faster

Context Size Estimation

Context size is estimated in "character units" which roughly correlate with tokens (approximately 4 characters per token for English text). This is a fast heuristic - actual token counts vary by model and content.

Truncation Strategies

Several truncation strategies are available:

  • :sliding_window - Keep most recent N messages (default)
  • :keep_system_user - Keep system prompt + first user message + recent
  • :summarize_old - Replace old messages with a summary (requires LLM call)

Usage

# Estimate context size
size = LemonAgent.Context.estimate_size(messages, system_prompt)

# Check if context is large
if LemonAgent.Context.large_context?(messages, system_prompt) do
  Logger.warning("Context is getting large")
end

# Truncate to fit budget
truncated = LemonAgent.Context.truncate(messages, max_chars: 100_000)

Telemetry Events

The following telemetry events are emitted:

  • [:lemon_agent, :context, :size] - Emitted when context size is measured

    • Measurements: %{char_count: integer, message_count: integer}
    • Metadata: %{has_system_prompt: boolean}

  • [:lemon_agent, :context, :warning] - Emitted when context exceeds threshold

    • Measurements: %{char_count: integer, threshold: integer}

    • Metadata: %{level: :warning | :critical}

Summary

Functions

check_size(messages, system_prompt \\ nil, opts \\ [])

Checks context size and emits warnings/telemetry if thresholds are exceeded.

estimate_size(messages, system_prompt \\ nil)

Estimates the size of the context in characters.

estimate_tokens(char_count)

Estimates the token count based on character count.

large_context?(messages, system_prompt \\ nil, opts \\ [])

Checks if the context is considered "large" (above warning threshold).

make_transform(opts \\ [])

Creates a transform_context function for use with AgentLoopConfig.

stats(messages, system_prompt \\ nil)

Returns context statistics for monitoring/debugging.

truncate(messages, opts \\ [])

Truncates message history to fit within limits.

Functions

check_size(messages, system_prompt \\ nil, opts \\ [])

@spec check_size([LemonAgent.Types.agent_message()], String.t() | nil, keyword()) ::
  :ok | :warning | :critical

Checks context size and emits warnings/telemetry if thresholds are exceeded.

This function should be called periodically (e.g., before each LLM call) to monitor context growth and emit early warnings.

Options

  • :warning_threshold - Chars for warning (default: 200000)
  • :critical_threshold - Chars for critical (default: 400000)
  • :log - Whether to log warnings (default: true)

Returns

  • :ok - Context is within normal limits
  • :warning - Context exceeds warning threshold
  • :critical - Context exceeds critical threshold

Examples

case LemonAgent.Context.check_size(messages, system_prompt) do
  :ok -> :continue
  :warning -> Logger.info("Consider truncating context")
  :critical -> truncate_context()
end

estimate_size(messages, system_prompt \\ nil)

@spec estimate_size([LemonAgent.Types.agent_message()], String.t() | nil) ::
  non_neg_integer()

Estimates the size of the context in characters.

This is a fast heuristic that counts characters in all message content. For rough token estimation, divide by 4.

Parameters

  • messages - List of agent messages
  • system_prompt - Optional system prompt string

Returns

Integer count of estimated characters.

Examples

iex> messages = [%{content: "Hello"}, %{content: "World"}]
iex> LemonAgent.Context.estimate_size(messages, "Be helpful")
20  # "Hello" + "World" + "Be helpful"

estimate_tokens(char_count)

@spec estimate_tokens(non_neg_integer()) :: non_neg_integer()

Estimates the token count based on character count.

Uses a conservative estimate of 4 characters per token. Actual token counts vary by model, language, and content type.

Examples

iex> LemonAgent.Context.estimate_tokens(4000)
1000

large_context?(messages, system_prompt \\ nil, opts \\ [])

@spec large_context?([LemonAgent.Types.agent_message()], String.t() | nil, keyword()) ::
  boolean()

Checks if the context is considered "large" (above warning threshold).

A large context may indicate memory pressure or approaching model limits. Consider truncation when this returns true.

Options

  • :threshold - Custom threshold in characters (default: 200000)

Examples

iex> LemonAgent.Context.large_context?(messages, "System prompt")
false

iex> LemonAgent.Context.large_context?(huge_messages, "Prompt", threshold: 1000)
true

make_transform(opts \\ [])

@spec make_transform(keyword()) :: ([LemonAgent.Types.agent_message()],
                              reference()
                              | nil ->
                                {:ok, [LemonAgent.Types.agent_message()]})

Creates a transform_context function for use with AgentLoopConfig.

This wraps the truncation logic in a function suitable for the transform_context configuration option.

Options

Same as truncate/2, plus:

  • :warn_on_truncation - Log when truncation occurs (default: true)

Examples

config = %AgentLoopConfig{
  transform_context: LemonAgent.Context.make_transform(max_messages: 50),
  ...
}

stats(messages, system_prompt \\ nil)

@spec stats([LemonAgent.Types.agent_message()], String.t() | nil) :: map()

Returns context statistics for monitoring/debugging.

Returns

A map with:

  • :message_count - Number of messages
  • :char_count - Total characters
  • :estimated_tokens - Approximate token count
  • :by_role - Message counts per role

Examples

stats = LemonAgent.Context.stats(messages, system_prompt)
IO.inspect(stats)
# %{
#   message_count: 10,
#   char_count: 5000,
#   estimated_tokens: 1250,
#   by_role: %{user: 5, assistant: 4, tool_result: 1}
# }

truncate(messages, opts \\ [])

@spec truncate(
  [LemonAgent.Types.agent_message()],
  keyword()
) :: {[LemonAgent.Types.agent_message()], non_neg_integer()}

Truncates message history to fit within limits.

Uses a sliding window strategy by default, keeping the most recent messages. The first user message is always preserved to maintain conversation context.

Options

  • :max_messages - Maximum number of messages to keep (default: 100)
  • :max_chars - Maximum total character count (default: 500000)
  • :strategy - Truncation strategy (default: :sliding_window)
  • :keep_first_user - Keep the first user message (default: true)

Strategies

  • :sliding_window - Keep most recent messages within limits
  • :keep_bookends - Keep first and last N messages, drop middle

Returns

A tuple of {truncated_messages, dropped_count}.

Examples

{truncated, dropped} = LemonAgent.Context.truncate(messages, max_messages: 50)
IO.puts("Dropped #{dropped} messages")