Normandy.Telemetry.Request (normandy v2.2.0)

View Source

The [:normandy, :llm, :request] span: one per provider request made inside an [:normandy, :agent, :llm_call] span, so retries, JSON parse retries and the structured-outputs fallback show up individually.

Who emits it

A third-party Normandy.Agents.Model emits no request spans unless it calls span/5 itself with the :request_tracker it receives in converse/3's opts. The llm_call stop's attempts then falls back to the number of times BaseAgent called converse/3.

Counting across layers

BaseAgent creates a tracker (new/1) per llm_call and passes it to Model.converse/3 as the :request_tracker option; the adapter hands it to the JSON retry loop, which passes it back with request_kind: :json_retry. The tracker is an explicit value (a :counters array), never process state, so BaseAgent's retries (outside the adapter) and the adapter's own extra requests share one 1-based attempt sequence.

Metadata

Start: :model, :provider (when known), :stream, :attempt, :request_kind (:initial | :retry | :json_retry | :structured_fallback; not :kind, which the exception event uses for the error class), and the llm_call's ids (:turn_id, :session_id, :conversation_id) and :agent_name when the tracker carries them. Stop adds :status (:ok), :response_id, :finish_reason (a string) and this request's normalized token counts (Normandy.Telemetry.Usage), each when known. A failed request emits :exception with kind: :error and the %Normandy.LLMError{} as :reason (stacktrace: [] when the request returned the error rather than raising).

The llm_call stop gets summary/1: :attempts, :json_retries, :structured_fallback.

Summary

Types

kind()
outcome()
t()

Functions

new(metadata \\ %{})

A tracker for one llm_call; metadata (nil values dropped) is added to every request span's start.

note_call(arg1)

Notes that the caller is about to call converse/3 (or start a stream attempt) once more. The first request of any call after the first is a :retry.

span(tracker, kind, metadata, fun, outcome)

Runs fun (one provider request) in a request span and returns its result unchanged.

stop_metadata(response_id, finish_reason, usage)

Stop metadata from a response's id, stop reason (converted to a string) and raw usage (normalized); missing values are left out.

summary(request)

:attempts, :json_retries and :structured_fallback for the llm_call stop. attempts is the number of requests, or of converse calls when the client emitted no request spans.

Types

kind()

@type kind() :: :initial | :retry | :json_retry | :structured_fallback

outcome()

@type outcome() :: {:ok, map()} | {:error, Normandy.LLMError.t()}

t()

@type t() :: %Normandy.Telemetry.Request{
  counters: :counters.counters_ref(),
  metadata: map()
}

Functions

new(metadata \\ %{})

@spec new(map()) :: t()

A tracker for one llm_call; metadata (nil values dropped) is added to every request span's start.

note_call(arg1)

@spec note_call(t() | nil) :: :ok

Notes that the caller is about to call converse/3 (or start a stream attempt) once more. The first request of any call after the first is a :retry.

span(tracker, kind, metadata, fun, outcome)

@spec span(t() | nil, kind() | nil, map(), (-> result), (result -> outcome())) ::
  result
when result: term()

Runs fun (one provider request) in a request span and returns its result unchanged.

kind is :json_retry or :structured_fallback when the caller knows it, else nil (:initial, or :retry once note_call/1 counted a second call). metadata holds :model, :provider, :stream. outcome maps the result to {:ok, stop_metadata} (see stop_metadata/3) or {:error, %LLMError{}}.

stop_metadata(response_id, finish_reason, usage)

@spec stop_metadata(String.t() | nil, atom() | String.t() | nil, map() | nil) :: map()

Stop metadata from a response's id, stop reason (converted to a string) and raw usage (normalized); missing values are left out.

summary(request)

@spec summary(t()) :: %{
  attempts: non_neg_integer(),
  json_retries: non_neg_integer(),
  structured_fallback: boolean()
}

:attempts, :json_retries and :structured_fallback for the llm_call stop. attempts is the number of requests, or of converse calls when the client emitted no request spans.