Normandy.Telemetry.Usage (normandy v2.2.0)

View Source

Normalizes a provider token-usage map into the token counts telemetry reports.

Anthropic reports input_tokens as the uncached remainder only: cache reads (cache_read_input_tokens) and cache writes (cache_creation_input_tokens) are separate fields. OpenTelemetry's GenAI conventions want gen_ai.usage.input_tokens to include all of them, so normalize/1 sums them.

The result uses these Normandy-side keys (all optional):

  • :input_tokens - input + cache read + cache write, summed over the fields that are present
  • :output_tokens
  • :cache_read_input_tokens
  • :cache_write_input_tokens - Anthropic's cache_creation_input_tokens

A count the provider did not report (missing or nil) is left out of the result, never 0 or nil. When it reports no count at all, the result is %{}.

The input may be a map or struct with atom keys, string keys, or a mix (Claudio's non-streaming usage uses atoms; streamed usage arrives with string keys). When it carries a non-empty iterations list (Claudio's refusal :fallbacks and compaction record every attempt there while the top-level counts cover only the returned one), the billed totals are the sum over the iterations and the top-level counts are ignored.

OpenAI-shaped usage (prompt_tokens, completion_tokens, optionally prompt_tokens_details.cached_tokens) is recognized too, when the map has no input_tokens: prompt_tokens already includes cached tokens, so it is the :input_tokens total as is, completion_tokens is :output_tokens, and cached_tokens (when present) is :cache_read_input_tokens. OpenAI reports no cache writes.

The raw usage map on [:normandy, :agent, :llm_call, :stop] is unchanged; these keys are added beside it.

Summary

Types

normalized()

Functions

normalize(usage)

Returns the normalized token counts for usage, or %{} when usage is not a map or reports no counts.

put_raw(metadata, usage)

Puts the raw provider usage under :usage in metadata when it is a non-empty map, and returns metadata unchanged otherwise, so a call whose provider reported no usage has no :usage key (never usage: nil).

sum(usage, usage)

Adds two raw provider usage maps (or structs), the one rule for combining the usage of several calls. A nil side returns the other (as a map). Keys are atoms in the result, so a string-keyed (wire) side adds to an atom-keyed one. Integer counts are summed and any other value is the later side's. When either side carries iterations, the result's iterations are both sides' entries in order (a side without them contributes one entry built from its top-level counts), so normalize/1 bills their sum and not the top-level counts.

Types

normalized()

@type normalized() :: %{
  optional(:input_tokens) => non_neg_integer(),
  optional(:output_tokens) => non_neg_integer(),
  optional(:cache_read_input_tokens) => non_neg_integer(),
  optional(:cache_write_input_tokens) => non_neg_integer()
}

Functions

normalize(usage)

@spec normalize(term()) :: normalized()

Returns the normalized token counts for usage, or %{} when usage is not a map or reports no counts.

Examples

iex> Normandy.Telemetry.Usage.normalize(%{
...>   input_tokens: 10,
...>   output_tokens: 5,
...>   cache_read_input_tokens: 100,
...>   cache_creation_input_tokens: 20
...> })
%{input_tokens: 130, output_tokens: 5, cache_read_input_tokens: 100, cache_write_input_tokens: 20}

iex> Normandy.Telemetry.Usage.normalize(nil)
%{}

put_raw(metadata, usage)

@spec put_raw(map(), term()) :: map()

Puts the raw provider usage under :usage in metadata when it is a non-empty map, and returns metadata unchanged otherwise, so a call whose provider reported no usage has no :usage key (never usage: nil).

Examples

iex> Normandy.Telemetry.Usage.put_raw(%{}, %{input_tokens: 3})
%{usage: %{input_tokens: 3}}

iex> Normandy.Telemetry.Usage.put_raw(%{}, nil)
%{}

sum(usage, usage)

@spec sum(map() | struct() | nil, map() | struct() | nil) :: map() | struct() | nil

Adds two raw provider usage maps (or structs), the one rule for combining the usage of several calls. A nil side returns the other (as a map). Keys are atoms in the result, so a string-keyed (wire) side adds to an atom-keyed one. Integer counts are summed and any other value is the later side's. When either side carries iterations, the result's iterations are both sides' entries in order (a side without them contributes one entry built from its top-level counts), so normalize/1 bills their sum and not the top-level counts.

Examples

iex> Normandy.Telemetry.Usage.sum(%{input_tokens: 3}, %{input_tokens: 4})
%{input_tokens: 7}

iex> Normandy.Telemetry.Usage.sum(nil, %{input_tokens: 4})
%{input_tokens: 4}