ExLLM.Core.Cost.Session (ex_llm v0.8.1)

View Source

Session-level cost tracking and aggregation functionality.

This module provides comprehensive cost tracking across an entire conversation session, allowing users to monitor cumulative costs, analyze usage patterns, and get detailed breakdowns by provider and model.

Usage

# Start a new session
session = ExLLM.Core.Cost.Session.new("chat_session_1")

# Add responses to track costs
session = session
  |> ExLLM.Core.Cost.Session.add_response(response1)
  |> ExLLM.Core.Cost.Session.add_response(response2)

# Get session summary
summary = ExLLM.Core.Cost.Session.get_summary(session)

# Format for display
formatted = ExLLM.Core.Cost.Session.format_summary(session, format: :detailed)

Features

  • Cumulative Cost Tracking: Track total costs across all messages
  • Token Aggregation: Monitor input/output token usage
  • Provider Breakdown: See costs by provider (OpenAI, Anthropic, etc.)
  • Model Breakdown: Analyze costs by specific model
  • Efficiency Metrics: Calculate cost per message, cost per token
  • Multiple Display Formats: Detailed, compact, and table formats

Summary

Functions

Add a response cost to the session tracking.

Format session cost summary for display.

Get session cost summary with detailed breakdown.

Get cost breakdown by model.

Initialize a new cost tracking session.

Get cost breakdown by provider.

Types

message_cost_entry()

@type message_cost_entry() :: %{
  timestamp: DateTime.t(),
  cost: float(),
  input_tokens: non_neg_integer(),
  output_tokens: non_neg_integer(),
  model: String.t(),
  provider: String.t()
}

model_stats()

@type model_stats() :: %{
  total_cost: float(),
  total_tokens: non_neg_integer(),
  message_count: non_neg_integer(),
  provider: String.t()
}

provider_stats()

@type provider_stats() :: %{
  total_cost: float(),
  total_tokens: non_neg_integer(),
  message_count: non_neg_integer()
}

session_summary()

@type session_summary() :: %{
  session_id: String.t(),
  duration: non_neg_integer(),
  total_cost: float(),
  total_tokens: non_neg_integer(),
  input_tokens: non_neg_integer(),
  output_tokens: non_neg_integer(),
  message_count: non_neg_integer(),
  average_cost_per_message: float(),
  cost_per_1k_tokens: float(),
  provider_breakdown: %{required(String.t()) => provider_stats()},
  model_breakdown: %{required(String.t()) => model_stats()}
}

t()

@type t() :: %ExLLM.Core.Cost.Session{
  messages: [message_cost_entry()],
  model_breakdown: %{required(String.t()) => model_stats()},
  provider_breakdown: %{required(String.t()) => provider_stats()},
  session_id: String.t(),
  start_time: DateTime.t(),
  total_cost: float(),
  total_input_tokens: non_neg_integer(),
  total_output_tokens: non_neg_integer()
}

Functions

add_response(session, response)

@spec add_response(t(), map()) :: t()

Add a response cost to the session tracking.

The response should contain cost information (typically from ExLLM.Core.Cost.calculate/3) and usage information.

Parameters

  • session - Current session state
  • response - LLM response with cost and usage data

Examples

session = session |> ExLLM.Core.Cost.Session.add_response(response)

format_summary(session, opts \\ [])

@spec format_summary(
  t(),
  keyword()
) :: String.t()

Format session cost summary for display.

Options

  • :format - Display format (:detailed, :compact, :table) (default: :detailed)

Examples

# Detailed format
ExLLM.Core.Cost.Session.format_summary(session)

# Compact format
ExLLM.Core.Cost.Session.format_summary(session, format: :compact)

# Table format
ExLLM.Core.Cost.Session.format_summary(session, format: :table)

get_summary(session)

@spec get_summary(t()) :: session_summary()

Get session cost summary with detailed breakdown.

Returns a comprehensive summary including totals, averages, and breakdowns.

Examples

summary = ExLLM.Core.Cost.Session.get_summary(session)
# => %{session_id: "...", total_cost: 0.45, ...}

model_breakdown(session)

@spec model_breakdown(t()) :: [map()]

Get cost breakdown by model.

Returns a list of models with their associated costs and usage statistics.

Examples

breakdown = ExLLM.Core.Cost.Session.model_breakdown(session)
# => [%{model: "gpt-4", total_cost: 0.30, ...}, ...]

new(session_id)

@spec new(String.t()) :: t()

Initialize a new cost tracking session.

Parameters

  • session_id - Unique identifier for the session

Examples

iex> session = ExLLM.Core.Cost.Session.new("chat_session_1")
iex> session.session_id
"chat_session_1"
iex> session.total_cost
0.0

provider_breakdown(session)

@spec provider_breakdown(t()) :: [map()]

Get cost breakdown by provider.

Returns a list of providers with their associated costs and usage statistics.

Examples

breakdown = ExLLM.Core.Cost.Session.provider_breakdown(session)
# => [%{provider: "openai", total_cost: 0.25, ...}, ...]