MLServe.Telemetry (MLServe v0.1.3)

Telemetry events emitted by MLServe.

MLServe treats instrumentation as a feature, not an afterthought: :telemetry is its only runtime dependency, and every prediction is wrapped in a span whether or not anyone is listening (an unattached event costs a single ETS lookup).

Attach MLServe.Telemetry.Logger.attach/1 for immediate visibility, or MLServe.Telemetry.Metrics.metrics/0 to feed Phoenix LiveDashboard.

Prediction span

MLServe.predict/3 and MLServe.batch_predict/3 emit the standard :telemetry.span/3 triple.

`[:ml_serve, :prediction, :start]`

Measurements
`system_time`	`System.system_time/0` at the start
`monotonic_time`	`System.monotonic_time/0` at the start
`batch_size`	`1` for `predict/3`, the list length for `batch_predict/3`

`[:ml_serve, :prediction, :stop]`

Measurements
`duration`	End-to-end, including queueing, hooks and cache
`queue_duration`	Time the request waited before a worker picked it up
`inference_duration`	Time inside the backend callback
`batch_size`	As above

queue_duration and inference_duration are 0 for shared-concurrency backends, which never queue — inference runs in the caller.

`[:ml_serve, :prediction, :exception]`

Measurements duration; metadata adds kind, reason and stacktrace. Emitted when the backend raises. A backend that returns {:error, reason} produces a :stop event with result: :error instead — an expected rejection is not an exception.

Prediction metadata

Key
`model`	model name
`version`	the version that actually served the request
`backend`	backend module
`batch?`	whether this came from `batch_predict/3`
`canary?`	whether canary routing chose this version
`cached?`	whether the result came from the cache (`:stop` only)
`result`	`:ok` or `:error` (`:stop` only)

version together with canary? is what makes a canary rollout decidable: your metrics backend can compare error rate and latency per version without any extra plumbing.

Lifecycle events

[:ml_serve, :model, :load] — measurements duration; metadata model, version, backend, workers, result.
[:ml_serve, :model, :unload] — measurements duration, drained (requests still in flight when the drain timeout expired; 0 is a clean drain); metadata model, version, backend.

These are single events carrying a duration rather than span triples, because a load either happened or it did not — there is no useful window to observe in between.

Cache and batching

[:ml_serve, :cache, :hit] / [:ml_serve, :cache, :miss] — measurements count: 1; metadata model, version.
[:ml_serve, :batch, :flush] — measurements size, wait_duration; metadata model, version, reason (:full or :timeout). A healthy dynamic-batching setup flushes mostly on :full; mostly :timeout means the batch window is longer than your traffic warrants.

Summary

Functions

events()

Returns every event name MLServe emits.

Functions

events()

@spec events() :: [[atom()]]

Returns every event name MLServe emits.

Useful for :telemetry.attach_many/4 and for tests.

Examples

iex> [:ml_serve, :prediction, :stop] in MLServe.Telemetry.events()
true