ALLM.Telemetry (allm v0.6.1)

Layer-B helper that wraps :telemetry.span/3 with ALLM-specific metadata defaults.

Every public Layer-C entry point (ALLM.generate/3, ALLM.stream_generate/3, ALLM.step/3, ALLM.stream_step/3, ALLM.chat/3, ALLM.stream/3, ALLM.generate_image/3 & siblings, ALLM.embed/3, ALLM.moderate/3, ALLM.classify/3, ALLM.synthesize/3, ALLM.transcribe/3, ALLM.stream_synthesize/3, ALLM.stream_synthesize_input/3, ALLM.stream_transcribe/3) is wrapped in a span. Attach :telemetry.attach_many/4 handlers to observe every execution.

Emitted events

Event name	When	Measurements	Metadata
`[:allm, :generate, :start]`	non-streaming generate enters	`system_time`, `monotonic_time`	`request_id`, `engine`, `model`
`[:allm, :generate, :stop]`	non-streaming generate exits	`duration`, `monotonic_time`	start metadata + `response`
`[:allm, :generate, :exception]`	closure raised	`duration`, `monotonic_time`	start metadata + `kind`, `reason`, `stacktrace`
`[:allm, :stream, :start \| :stop \| :exception]`	streaming generate	as above; `:stop` `response` is `nil` (lazy)	start metadata + `response: nil`
`[:allm, :step, :start \| :stop \| :exception]`	single chat step	as above	start metadata + `step_result`
`[:allm, :chat, :start \| :stop \| :exception]`	multi-turn chat loop	as above	start metadata + `chat_result`
`[:allm, :tool, :start \| :stop \| :exception]`	per-tool execution	as above	start metadata (`tool`, `tool_call`, `engine`, `model`, `request_id`) + `result` on `:stop`; a timed-out tool emits `:exception` with `duration: 0`
`[:allm, :image, :start \| :stop \| :exception]`	image generation	`duration`, plus `image_count` on `:stop`	`request_id`, `engine`, `model`, `operation`, `n`, plus `usage`, `response`, `error` on `:stop`
`[:allm, :embed, :start \| :stop \| :exception]`	text embedding	`duration`, plus `chunk_count` and `embedding_count` on `:stop`	`request_id`, `engine`, `model`, `input_count`, plus `usage`, `response`, `error` on `:stop`
`[:allm, :moderate, :start \| :stop \| :exception]`	content moderation	`duration`, plus `result_count` and `flagged_count` on `:stop`	`request_id`, `engine`, `model`, `input_count`, `multimodal`, plus `usage`, `response`, `error` on `:stop`
`[:allm, :classify, :start \| :stop \| :exception]`	typed classification	`duration`, plus `answer_count` on `:stop`	`request_id`, `engine`, `model`, `question_count`, plus `usage`, `response`, `error` on `:stop`
`[:allm, :synthesize, :start \| :stop \| :exception]`	text-to-speech	`duration`, plus `audio_bytes` on `:stop`	`request_id`, `engine`, `model`, `input_length`, plus `usage`, `response`, `error` on `:stop`
`[:allm, :transcribe, :start \| :stop \| :exception]`	speech-to-text	`duration`, plus `text_length` on `:stop`	`request_id`, `engine`, `model`, `audio_mime`, plus `usage`, `response`, `error` on `:stop`
`[:allm, :stream_synthesize, :start \| :stop \| :exception]`	streaming text-to-speech (`ALLM.stream_synthesize/3` and `ALLM.stream_synthesize_input/3`)	`duration`; `:stop` fires when the stream is returned, not when it drains	`request_id`, `engine`, `model`, `input_length` (`nil` for the input form), plus `response: nil` on `:stop`
`[:allm, :stream_transcribe, :start \| :stop \| :exception]`	streaming speech-to-text (`ALLM.stream_transcribe/3`)	as above	`request_id`, `engine`, `model`, `sample_rate`, plus `response: nil` on `:stop`
`[:allm, :audio, :first_chunk]`	once per audio stream, at its first `:audio_delta` (speech), or its first `:partial_transcript` or non-blank `:committed_transcript`, whichever comes first (transcription)	`latency` (native time units from the façade call)	`request_id`, `capability` (`:speech` or `:transcription`), `provider_model`
`[:allm, :adapter, :retry]`	per-attempt retry (non-streaming)	`system_time`	the calling façade's metadata (e.g. `request_id`, `model`) + `attempt`, `delay_ms`, `reason`

[:allm, :embed, :stop] carries embedding_count and chunk_count on both paths — each is 0 on the error path rather than absent, so the measurement key set is stable. chunk_count is the only signal that one ALLM.embed/3 call became N HTTP requests; see ALLM.embed/3 for the batching contract.

[:allm, :moderate, :stop] follows the same rule: result_count and flagged_count are present on both paths (0 each on error). It also carries usage: nil unconditionally, even though ALLM.ModerationResponse has no :usage field — a stable key set across capability spans is what a metrics backend wants, and it is what keeps a handler written against [:allm, :embed, :stop] from KeyErroring when pointed at :moderate. There is no chunk_count: ALLM.moderate/3 does not chunk (see its "Batching" section).

input_count on the :moderate span is length(request.input) — the raw element count, not the provider's item count, which is 1 whenever multimodal is true (an :input list carrying an ALLM.ImagePart is one multimodal item, judged together). The two agree exactly for an all-strings input; multimodal rides alongside so a consumer can derive the item count without a second measurement. A 40-element input with one ALLM.ImagePart therefore reports input_count: 40 and is still accepted by an adapter whose ALLM.ModerationAdapter.max_batch_size/0 is 32.

[:allm, :classify, :stop] follows the same stable-key rule: answer_count is 0 and usage is nil on the error path. question_count is the number of questions on the request (0 when :questions is not a plain map — a struct counts as 0), and model is the classification slot's model (request.model || engine.classification_model), never the chat engine.model — nil when neither is set.

[:allm, :synthesize, :stop] and [:allm, :transcribe, :stop] follow the same stable-key rule: audio_bytes / text_length are 0 on the error path, and usage is nil there. input_length is String.length/1 of the request's :input (0 when it is not a binary), and audio_mime is the request audio's :mime_type (nil when there is none). model is the audio slot's model (request.model || engine.speech_model / engine.transcription_model), never the chat engine.model, and is nil when neither is set — the adapter's own default applies after the span has started.

[:allm, :synthesize, :stop] metadata carries the audio. Its response is the ALLM.SpeechResponse, whose :audio holds the full synthesized bytes. A handler that logs or ships metadata.response wholesale moves megabytes per call; read the scalar fields you need instead (ALLM.Audio's Inspect implementation prints a byte count, not the payload, but a JSON encoder or :erlang.term_to_binary/1 does not). [:allm, :transcribe, :start] metadata does not carry the request audio at all — only its MIME type.

The streaming audio spans follow the chat :stream carve-out: :stop carries response: nil, because the span closes when the enumerable is returned, before any audio has arrived. [:allm, :audio, :first_chunk] is the event that measures time to first audio (or first transcript text, partial or committed), which is the latency a voice interface is judged by. provider_model is the :model on the stream's start event, or the dispatched request's model when the adapter reported none (nil when neither is known). Neither the spans nor the event carry audio bytes or transcript text.

Common metadata

Every span carries :request_id, :engine, and :model on :start, and the same on :stop (plus the per-span *_result / response / error extras). :request_id is generated at the outermost public call and threaded into nested calls via opts[:request_id], so per-tool spans inside a chat/3 loop share the parent chat's id.

Why request_id is metadata-only

:request_id lives on telemetry span metadata and on Response.request_id post-collection — it does NOT extend the ALLM.Event closed tagged-tuple union. A consumer who folds events by hand (without ALLM.StreamCollector) reads :request_id from the surrounding span context, not from individual event payloads.

Span nesting

Per-tool spans ([:allm, :tool, ...]) execute inside their parent step span ([:allm, :step, ...]); the parent step span itself nests inside its parent chat span when invoked from chat/3 / stream/3.

Exception handling

span/3 delegates exception trapping to :telemetry.span/3 which automatically catches raises in the closure, emits [:allm, name, :exception] with %{kind, reason, stacktrace} metadata, and re-raises the exception to the caller — preserving the existing bubble-up semantics of every wrapped function.

Summary

Types

common_metadata()

Common metadata attached to every Layer-C span.

span_name()

Span suffix; the prefix [:allm] is fixed.

Functions

event_prefix()

Return the fixed event-name prefix for every ALLM telemetry event.

execute(suffix_path, measurements, metadata)

Emit a single non-span event under the [:allm | suffix_path] event name. Used by ALLM.Retry for [:allm, :adapter, :retry]. No metadata merging or measurement injection — the caller supplies both maps in full.

request_id()

Generate a fresh 22-character URL-safe Base64 request id.

span(name, start_metadata, fun)

Wrap a closure in a :telemetry.span/3 call under the [:allm, name] prefix.

Types

common_metadata()

@type common_metadata() :: %{
  optional(:request_id) => String.t(),
  optional(:engine) => term(),
  optional(:model) => term(),
  optional(atom()) => term()
}

Common metadata attached to every Layer-C span.

span_name()

@type span_name() ::
  :generate
  | :stream
  | :step
  | :chat
  | :tool
  | :image
  | :embed
  | :moderate
  | :classify
  | :synthesize
  | :transcribe
  | :stream_synthesize
  | :stream_transcribe

Span suffix; the prefix [:allm] is fixed.

Functions

event_prefix()

@spec event_prefix() :: [:allm]

Return the fixed event-name prefix for every ALLM telemetry event.

Examples

iex> ALLM.Telemetry.event_prefix()
[:allm]

execute(suffix_path, measurements, metadata)

@spec execute([atom(), ...], map(), map()) :: :ok

Examples

iex> :telemetry.attach(
...> "doctest-execute",
...> [:allm, :doctest, :ping],
...> fn _n, _m, _md, %{owner: pid} -> send(pid, :got_event) end,
...> %{owner: self()}
...>)
iex> ALLM.Telemetry.execute([:doctest, :ping], %{count: 1}, %{tag: :doc})
:ok
iex> :telemetry.detach("doctest-execute")
iex> receive do
...> :got_event -> :ok
...> after
...> 100 -> :no_event
...> end
:ok

request_id()

@spec request_id() :: String.t()

Generate a fresh 22-character URL-safe Base64 request id.

Sourced from 16 cryptographic random bytes. Used to correlate the start, stop, and any exception events of a single Layer-C call. The outermost public function generates the id; inner functions inherit via opts[:request_id].

Examples

iex> id = ALLM.Telemetry.request_id()
iex> byte_size(id)
22
iex> id =~ ~r/^[A-Za-z0-9_-]{22}$/
true

span(name, start_metadata, fun)

@spec span(
  span_name(),
  common_metadata(),
  (-> {result, map()} | {result, map(), map()})
) :: result
when result: var

Wrap a closure in a :telemetry.span/3 call under the [:allm, name] prefix.

The closure must return either:

{result, stop_metadata_extras} — the 2-tuple form (default, used by :generate | :stream | :step | :chat | :tool spans). stop_metadata_extras is shallow-merged on top of the start metadata at :stop emission time so per-span extras (:response, :step_result, :chat_result, :result) appear alongside the common keys.
{result, extra_measurements, stop_metadata_extras} — the 3-tuple form, for spans that inject custom :stop measurements beyond :duration and :monotonic_time. :image, :embed, :moderate, :synthesize, :transcribe and :classify spans use this form to carry :image_count / :embedding_count + :chunk_count / :result_count + :flagged_count / :audio_bytes / :text_length / :answer_count as measurements (numeric metrics → measurements; structured context → metadata).

Caller-supplied start_metadata is forwarded to the :start event unchanged and used as the base for the :stop event's metadata.

Carve-out: `:stream :stop` `:response` is `nil`

Per ALLM.StreamRunner.run/3, the :stream span's :stop metadata carries :response => nil — materialising the wrapped enumerable to populate the %Response{} would defeat consumer-driven laziness. Consumers needing the canonical response should either fold the returned stream themselves (ALLM.StreamCollector.to_response/1) or call ALLM.generate/3, whose :generate :stop DOES carry the reduced %Response{}.

Examples

iex> :telemetry.attach(
...> "doctest-handler",
...> [:allm, :generate, :stop],
...> fn _name, _measurements, %{result: r}, %{owner: pid} ->
...> send(pid, {:doctest_event, r})
...> end,
...> %{owner: self()}
...>)
iex> ALLM.Telemetry.span(:generate, %{request_id: "x"}, fn ->
...> {:done, %{result: :ok}}
...> end)
:done
iex> :telemetry.detach("doctest-handler")
iex> receive do
...> {:doctest_event, payload} -> payload
...> after
...> 100 -> :no_event
...> end
:ok

ALLM.Telemetry (allm v0.6.1)

Emitted events

Common metadata

Why request_id is metadata-only

Span nesting

Exception handling

Summary

Types

Functions

Types

common_metadata()

span_name()

Functions

event_prefix()

Examples

execute(suffix_path, measurements, metadata)

Examples

request_id()

Examples

span(name, start_metadata, fun)

Carve-out: :stream :stop :response is nil

Examples

Carve-out: `:stream :stop` `:response` is `nil`