Legion.Telemetry (Legion v0.5.0)

View Source

Telemetry integration for Legion agents.

Legion emits the following telemetry events:

Agent Lifecycle Events

  • [:legion, :agent, :started] — emitted during init/1, before any persisted conversation is restored

    • Measurements: %{system_time: NaiveDateTime.t()}
    • Metadata: %{agent: module, agent_id: String.t()} (plus parent_agent_id: String.t() when the agent was started inside another agent's run)
    • agent_id names the conversation — stable across restarts, so a resumed conversation emits under the same id.
    • Not emitted if init/1 crashes before the event fires (e.g. while building the system prompt) — in that case :stopped is not emitted either, since GenServer does not call terminate/2 on init failure.
  • [:legion, :agent, :stopped] — agent process terminated via terminate/2

    • Measurements: %{system_time: NaiveDateTime.t()}
    • Metadata: %{agent: module, agent_id: String.t()} (plus parent_agent_id: String.t() when the agent was started inside another agent's run)

Agent Message Events (spans)

  • [:legion, :agent, :message, :start | :stop | :exception] — agent handling a message

    • Metadata includes: agent, agent_id, message
    • Stop adds: iterations (count of assistant turns in this message), status (:ok or :cancel), result (the value returned, or the cancellation reason such as :reached_max_iterations), and bindings (the variable bindings carried out of the turn)

Iteration Events (spans)

  • [:legion, :iteration, :start | :stop | :exception]

    • Metadata includes: agent, agent_id, iteration
    • Stop adds: action

LLM Request Events (spans)

  • [:legion, :llm, :request, :start | :stop | :exception]

    • Metadata includes: agent, agent_id, model, message_count, iteration
    • Stop adds: object or error, and usage (the string-keyed usage map with its "at" timestamp, when a response was received)

Sandbox Eval Events (spans)

  • [:legion, :sandbox, :eval, :start | :stop | :exception]

    • Metadata includes: agent, agent_id, code
    • Stop adds: success and result or error.

Eval Guard Events

  • [:legion, :eval_guard, :denied] — a guard refused generated code
    • Metadata: %{agent: module, agent_id: String.t(), guard: module, code: String.t(), reason: String.t()}

Rate Limit Events

  • [:legion, :rate_limit, :exceeded] — a rate limiter denied a turn before it started; metadata carries the identity and policy of the rule that denied it, the usage measured for it, and the violations
    • Measurements: %{system_time: NaiveDateTime.t()}
    • Metadata: %{agent: module, agent_id: String.t(), identity: map, policy: Legion.RateLimiter.Policy.t(), usage: map, violations: [atom]}
    • violations names the limits that were reached, e.g. [:max_tokens].

Default Logger

A default logger is provided that outputs human-readable telemetry to Logger. Attach it with Legion.Telemetry.attach_default_logger/1.

Summary

Functions

Attaches a default logger for Legion telemetry events.

Detaches the default logger.

Emits a single telemetry event. Injects agent_id from the process dictionary.

Wraps a function with :start / :stop / :exception telemetry events.

Functions

attach_default_logger(opts \\ [])

Attaches a default logger for Legion telemetry events.

Options

  • :level — log level, defaults to :info
  • :events:all or a list of event categories (:agent, :message, :iteration, :llm, :sandbox). Defaults to :all.

detach_default_logger()

Detaches the default logger.

emit(event, measurements \\ %{}, metadata)

Emits a single telemetry event. Injects agent_id from the process dictionary.

span(event_prefix, metadata, fun)

Wraps a function with :start / :stop / :exception telemetry events.

The function should return {result, extra_stop_metadata}. agent_id is automatically injected from the process dictionary.