Dsxir.Telemetry (dsxir v0.2.0)

Copy Markdown

Event-name constants and the emit/3 helper for dsxir telemetry.

Documents the full event vocabulary and exposes a thin wrapper over :telemetry.execute/3. Producers (predictors, adapters, optimizers, evaluate) and subscribers (the Dsxir.History ETS handler, the upstream LM relay, user-supplied handlers) attach as their consumers land.

Event vocabulary

[:dsxir, :predictor, :start]
[:dsxir, :predictor, :stop]
[:dsxir, :predictor, :exception]

[:dsxir, :adapter, :format]
[:dsxir, :adapter, :parse]
[:dsxir, :adapter, :fallback]

[:dsxir, :optimizer, :start]
[:dsxir, :optimizer, :stop]
[:dsxir, :optimizer, :trial]
[:dsxir, :optimizer, :item_error]
[:dsxir, :optimizer, :insufficient_demos]

[:dsxir, :miprov2, :proposer]
[:dsxir, :miprov2, :rerank]

[:dsxir, :evaluate, :item]
[:dsxir, :evaluate, :stop]

[:dsxir, :runtime_program, :skipped]

Always-present keys

The :error_class metadata key on [:dsxir, :predictor, :stop] and [:dsxir, :predictor, :exception] is always present, with value nil when no error occurred. Subscribers should branch on nil, never on Map.has_key?/2.

Token measurements (tokens_in, tokens_out, cost) follow the same always-present-nil convention on [:dsxir, :predictor, :stop]. They are populated by Dsxir.LM implementations via the Dsxir.LM.usage() shape; when the upstream provider did not report usage, the impl returns Dsxir.LM.empty_usage/0 (all three keys nil).

Optimizer events

[:dsxir, :optimizer, :start]

  • Measurements: %{system_time: integer()}.
  • Metadata: %{optimizer: module(), trainset_size: non_neg_integer(), error_class: nil}.

[:dsxir, :optimizer, :stop]

  • Measurements: %{duration: integer(), score: nil | float()}. score is nil for optimizers that do not compute a holdout score during compile (e.g. Dsxir.Optimizer.LabeledFewShot, Dsxir.Optimizer.BootstrapFewShot); future optimizers (MIPROv2, GEPA) may populate it.

  • Metadata: %{optimizer: module(), trainset_size: non_neg_integer(), error_class: nil | atom()}. error_class is nil on success and the aggregated error class atom on failure.

[:dsxir, :optimizer, :trial]

  • Measurements: %{score: float()}. Coerced metric value for the trial.
  • Metadata: %{optimizer: module(), round: pos_integer(), example_index: non_neg_integer(), kept: boolean(), error_class: nil}.

[:dsxir, :optimizer, :item_error]

  • Measurements: %{system_time: integer()}.
  • Metadata: %{optimizer: module(), round: pos_integer(), example_index: non_neg_integer(), error: Exception.t(), error_class: atom()}.

[:dsxir, :optimizer, :insufficient_demos]

  • Measurements: %{requested: non_neg_integer(), got: non_neg_integer()}.
  • Metadata: %{predictor: atom(), requested: non_neg_integer(), got: non_neg_integer(), reached_examples: non_neg_integer(), total_examples: non_neg_integer(), degraded_excluded: boolean()}.

Emitted once per predictor whose final bootstrapped pool has fewer than max_bootstrapped_demos entries. Never raises; subscribers may use the event to surface trainset-coverage gaps without aborting compilation.

Subscribers branch on nil for score and error_class; never on Map.has_key?/2.

Runtime-program events

[:dsxir, :runtime_program, :skipped]

  • Measurements: %{}.
  • Metadata: %{node: atom(), reason: :guard_false | :upstream_required_skipped, program_id: term(), program_version: term()}.

Emitted by the runtime-program executor each time a graph node is skipped, either because a declared guard returned false or because an upstream required-input dependency was itself skipped. Documented here; produced by the executor.

Summary

Functions

Event name for [:dsxir, :adapter, :fallback].

Event name for [:dsxir, :adapter, :format].

Event name for [:dsxir, :adapter, :parse].

Event name for [:dsxir, :evaluate, :item].

Event name for [:dsxir, :evaluate, :stop].

Event name for [:dsxir, :miprov2, :proposer] (point-in-time proposer outcome).

Event name for [:dsxir, :miprov2, :rerank] (point-in-time full-eval rerank).

Event name for [:dsxir, :optimizer, :item_error].

Event name for [:dsxir, :optimizer, :start].

Event name for [:dsxir, :optimizer, :stop].

Event name for [:dsxir, :optimizer, :trial].

Event name for [:dsxir, :predictor, :exception].

Event name for [:dsxir, :predictor, :start].

Event name for [:dsxir, :predictor, :stop].

Types

event()

@type event() :: [atom(), ...]

Functions

adapter_fallback()

@spec adapter_fallback() :: event()

Event name for [:dsxir, :adapter, :fallback].

adapter_format()

@spec adapter_format() :: event()

Event name for [:dsxir, :adapter, :format].

adapter_parse()

@spec adapter_parse() :: event()

Event name for [:dsxir, :adapter, :parse].

emit(event, measurements \\ %{}, metadata \\ %{})

@spec emit(event(), map(), map()) :: :ok

Thin wrapper over :telemetry.execute/3.

evaluate_item()

@spec evaluate_item() :: event()

Event name for [:dsxir, :evaluate, :item].

evaluate_stop()

@spec evaluate_stop() :: event()

Event name for [:dsxir, :evaluate, :stop].

miprov2_proposer()

@spec miprov2_proposer() :: event()

Event name for [:dsxir, :miprov2, :proposer] (point-in-time proposer outcome).

miprov2_rerank()

@spec miprov2_rerank() :: event()

Event name for [:dsxir, :miprov2, :rerank] (point-in-time full-eval rerank).

optimizer_item_error()

@spec optimizer_item_error() :: event()

Event name for [:dsxir, :optimizer, :item_error].

optimizer_start()

@spec optimizer_start() :: event()

Event name for [:dsxir, :optimizer, :start].

optimizer_stop()

@spec optimizer_stop() :: event()

Event name for [:dsxir, :optimizer, :stop].

optimizer_trial()

@spec optimizer_trial() :: event()

Event name for [:dsxir, :optimizer, :trial].

predictor_exception()

@spec predictor_exception() :: event()

Event name for [:dsxir, :predictor, :exception].

predictor_start()

@spec predictor_start() :: event()

Event name for [:dsxir, :predictor, :start].

predictor_stop()

@spec predictor_stop() :: event()

Event name for [:dsxir, :predictor, :stop].