Context window management for conversation history.
Provides token estimation and window trimming helpers used by
Candil.Conversation to stay within model context limits.
Summary
Functions
Default max response tokens.
Estimates the tokens in a text string.
Estimates the tokens in a single message.
Estimates tokens for any input.
Estimates the total token count for a conversation's messages.
Trims old messages from a list while preserving the system prompt.
Functions
@spec default_max_response_tokens() :: pos_integer()
Default max response tokens.
@spec estimate_content_tokens(binary()) :: non_neg_integer()
Estimates the tokens in a text string.
@spec estimate_message_tokens(map()) :: non_neg_integer()
Estimates the tokens in a single message.
@spec estimate_tokens(binary()) :: non_neg_integer()
Estimates tokens for any input.
@spec token_estimate([map()], String.t() | nil, map()) :: non_neg_integer()
Estimates the total token count for a conversation's messages.
@spec trim_to_context([map()], String.t() | nil, non_neg_integer()) :: [map()]
Trims old messages from a list while preserving the system prompt.