Candil.Conversation (Candil v3.0.0)

Conversation history management for Candil.Llm.

Maintains a message history and automatically manages context window limits. When the accumulated token estimate exceeds max_context_tokens, older messages are trimmed while always preserving the system prompt.

Token estimation is delegated to Candil.Conversation.Context.

Usage

conv = Candil.Conversation.new(
  model: :llama3,
  system: "You are a helpful Elixir assistant.",
  max_context_tokens: 4096
)

{:ok, conv, response} = Candil.Conversation.chat(conv, "What is a GenServer?")
{:ok, conv, response} = Candil.Conversation.chat(conv, "Give me a code example.")

IO.puts(response.content)

Summary

Types

message()

t()

Functions

add_message(conv, role, content)

Appends a message to the history without calling the model.

available_context_tokens(conv)

Returns the available context tokens (accounting for max_response_tokens).

chat(conv, user_message)

Sends a user message and returns {:ok, updated_conv, response}.

messages(conversation)

Returns the full message list including the system prompt as the first message (if set).

new(opts)

Creates a new conversation.

reset(conv)

Resets the conversation history, keeping the system prompt and config.

token_estimate(conv)

Returns the approximate token count for the current history.

turn_count(conversation)

Returns the number of turns (user+assistant pairs) in the conversation.

Types

message()

@type message() :: Candil.Inference.message()

t()

@type t() :: %Candil.Conversation{
  max_context_tokens: pos_integer(),
  max_response_tokens: pos_integer(),
  messages: [message()],
  model: atom() | Candil.Model.t(),
  opts: keyword(),
  provider: Candil.Provider.t() | nil,
  system: binary() | nil
}

Functions

add_message(conv, role, content)

@spec add_message(t(), binary(), binary()) :: t()

Appends a message to the history without calling the model.

Useful for agent loops that manage their own backend calls. Returns the updated conversation.

available_context_tokens(conv)

@spec available_context_tokens(t()) :: non_neg_integer()

Returns the available context tokens (accounting for max_response_tokens).

chat(conv, user_message)

@spec chat(t(), binary()) :: {:ok, t(), Candil.Inference.response()} | {:error, any()}

Sends a user message and returns {:ok, updated_conv, response}.

Appends the user message to history, calls the model, appends the assistant response, and trims history if needed.

messages(conversation)

@spec messages(t()) :: [message()]

Returns the full message list including the system prompt as the first message (if set).

new(opts)

@spec new(keyword()) :: t()

Creates a new conversation.

Options

:model — atom alias (local engine) or Candil.Model struct (required)
:provider — Candil.Provider struct for remote models
:system — system prompt (default: nil)
:max_context_tokens — approximate token limit for history (default: 4096)
:max_response_tokens — max tokens to generate in responses (default: 512)

reset(conv)

@spec reset(t()) :: t()

Resets the conversation history, keeping the system prompt and config.

token_estimate(conv)

@spec token_estimate(t()) :: non_neg_integer()

Returns the approximate token count for the current history.

turn_count(conversation)

@spec turn_count(t()) :: non_neg_integer()

Returns the number of turns (user+assistant pairs) in the conversation.