Conversation history management for Candil.Llm.
Maintains a message history and automatically manages context window
limits. When the accumulated token estimate exceeds max_context_tokens,
older messages are trimmed while always preserving the system prompt.
Usage
conv = Candil.Conversation.new(
model: :llama3,
system: "You are a helpful Elixir assistant.",
max_context_tokens: 4096
)
{:ok, conv, response} = Candil.Conversation.chat(conv, "What is a GenServer?")
{:ok, conv, response} = Candil.Conversation.chat(conv, "Give me a code example.")
IO.puts(response.content)Remote provider
conv = Candil.Conversation.new(
model: gpt4o_model,
provider: openai_provider,
system: "You are a code reviewer.",
max_context_tokens: 16_000
)Token estimation
Token counts are estimated as ceil(char_count / 4). This is an
approximation — actual counts vary by model. For precise limits, use a
tokeniser specific to your model.
Summary
Functions
Sends a user message and returns {:ok, updated_conv, response}.
Returns the full message list including the system prompt as the first message (if set).
Creates a new conversation.
Resets the conversation history, keeping the system prompt and config.
Returns the approximate token count for the current history.
Returns the number of turns (user+assistant pairs) in the conversation.
Types
@type message() :: Candil.Inference.message()
@type t() :: %Candil.Conversation{ max_context_tokens: pos_integer(), messages: [message()], model: atom() | Candil.Model.t(), opts: keyword(), provider: Candil.Provider.t() | nil, system: binary() | nil }
Functions
@spec chat(t(), binary()) :: {:ok, t(), Candil.Inference.response()} | {:error, any()}
Sends a user message and returns {:ok, updated_conv, response}.
Appends the user message to history, calls the model, appends the assistant response, and trims history if needed.
Returns the full message list including the system prompt as the first message (if set).
Creates a new conversation.
Options
:model— atom alias (local engine) orCandil.Modelstruct (required):provider—Candil.Providerstruct for remote models:system— system prompt (default:nil):max_context_tokens— approximate token limit for history (default:4096)- Any other options are forwarded to
Candil.chat/3on each turn (:temperature,:max_tokens, etc.)
Resets the conversation history, keeping the system prompt and config.
@spec token_estimate(t()) :: non_neg_integer()
Returns the approximate token count for the current history.
@spec turn_count(t()) :: non_neg_integer()
Returns the number of turns (user+assistant pairs) in the conversation.