Dsxir. Telemetry
(dsxir v0.4.0)
Copy Markdown
Event-name constants and the emit/3 helper for dsxir telemetry.
Documents the full event vocabulary and exposes a thin wrapper over
:telemetry.execute/3. Producers (predictors, adapters, optimizers, evaluate)
and subscribers (the Dsxir.History ETS handler, the upstream LM relay,
user-supplied handlers) attach as their consumers land.
Event vocabulary
[:dsxir, :predictor, :start]
[:dsxir, :predictor, :stop]
[:dsxir, :predictor, :exception]
[:dsxir, :predictor, :code_exec, :attempt]
[:dsxir, :predictor, :best_of_n | :refine, :attempt]
[:dsxir, :predictor, :best_of_n | :refine, :stop]
[:dsxir, :predictor, :ensemble, :member]
[:dsxir, :predictor, :ensemble, :stop]
[:dsxir, :adapter, :format]
[:dsxir, :adapter, :parse]
[:dsxir, :adapter, :fallback]
[:dsxir, :optimizer, :start]
[:dsxir, :optimizer, :stop]
[:dsxir, :optimizer, :trial]
[:dsxir, :optimizer, :item_error]
[:dsxir, :optimizer, :insufficient_demos]
[:dsxir, :miprov2, :proposer]
[:dsxir, :miprov2, :rerank]
[:dsxir, :evaluate, :item]
[:dsxir, :evaluate, :stop]
[:dsxir, :lm, :embed, :stop]
[:dsxir, :runtime_program, :skipped]Always-present keys
The :error_class metadata key on [:dsxir, :predictor, :stop] and
[:dsxir, :predictor, :exception] is always present, with value nil when no
error occurred. Subscribers should branch on nil, never on Map.has_key?/2.
Token measurements on [:dsxir, :predictor, :stop] follow the same
always-present-nil convention: tokens_in, tokens_out,
cache_read_tokens, cache_write_tokens, reasoning_tokens, and cost
are always present in measurements; their values are nil when the upstream
LM did not report them. Dsxir.LM implementations return a %Dsxir.Cost{}
(or Dsxir.LM.empty_usage/0 = Dsxir.Cost.zero/0 when nothing was
reported); Dsxir.Cost.to_measurements/1 flattens it into these keys.
The [:dsxir, :predictor, :stop] metadata always carries two additional
keys: cost: (a %Dsxir.Cost{} struct) and _cost_scope: (a list of
active Dsxir.Cost.track/1 scope ids, [] when not inside a track block).
Subscribers can branch on both keys unconditionally. Dsxir.Cost.track/1
uses _cost_scope to attribute costs from nested predictor calls back to
the enclosing block, returning {result, %Dsxir.Cost{}} with the summed
cost of all predictor-stop events fired in scope.
Embedding events
[:dsxir, :lm, :embed, :stop]
- Measurements:
%{duration, tokens_in, tokens_out, cache_read_tokens, cache_write_tokens, reasoning_tokens, cost}— same always-present-nil flattening as predictor stops, viaDsxir.Cost.to_measurements/1. - Metadata:
%{model, cost: %Dsxir.Cost{}, _cost_scope: [...]}merged over the resolvedDsxir.Settings:metadataframe. Emitted by theDsxir.LM.embed/2dispatcher on a successful embed only (never on error).
Optimizer events
[:dsxir, :optimizer, :start]
- Measurements:
%{system_time: integer()}. - Metadata:
%{optimizer: module(), trainset_size: non_neg_integer(), error_class: nil}.
[:dsxir, :optimizer, :stop]
Measurements:
%{duration: integer(), score: nil | float()}.scoreisnilfor optimizers that do not compute a holdout score during compile (e.g.Dsxir.Optimizer.LabeledFewShot,Dsxir.Optimizer.BootstrapFewShot); future optimizers (MIPROv2, GEPA) may populate it.- Metadata:
%{optimizer: module(), trainset_size: non_neg_integer(), error_class: nil | atom()}.error_classisnilon success and the aggregated error class atom on failure.
[:dsxir, :optimizer, :trial]
- Measurements:
%{score: float()}. Coerced metric value for the trial. - Metadata:
%{optimizer: module(), round: pos_integer(), example_index: non_neg_integer(), kept: boolean(), error_class: nil}.
[:dsxir, :optimizer, :item_error]
- Measurements:
%{system_time: integer()}. - Metadata:
%{optimizer: module(), round: pos_integer(), example_index: non_neg_integer(), error: Exception.t(), error_class: atom()}.
[:dsxir, :optimizer, :insufficient_demos]
- Measurements:
%{requested: non_neg_integer(), got: non_neg_integer()}. - Metadata:
%{predictor: atom(), requested: non_neg_integer(), got: non_neg_integer(), reached_examples: non_neg_integer(), total_examples: non_neg_integer(), degraded_excluded: boolean()}.
Emitted once per predictor whose final bootstrapped pool has fewer than
max_bootstrapped_demos entries. Never raises; subscribers may use the event
to surface trainset-coverage gaps without aborting compilation.
Subscribers branch on nil for score and error_class; never on
Map.has_key?/2.
Runtime-program events
[:dsxir, :runtime_program, :skipped]
- Measurements:
%{}. Metadata:
%{node: atom(), reason: :guard_false | :upstream_required_skipped, program_id: term(), program_version: term()}.
Emitted by the runtime-program executor each time a graph node is skipped,
either because a declared guard returned false or because an upstream
required-input dependency was itself skipped. Documented here; produced by
the executor.
Sampling-predictor events
Dsxir.Predictor.BestOfN and Dsxir.Predictor.Refine share the
Dsxir.Predictor.Sampling core, which emits a parameterized event whose
third segment is :best_of_n or :refine. Build the names via
sampling_attempt/1 and sampling_stop/1.
[:dsxir, :predictor, :best_of_n | :refine, :attempt]
Measurements:
%{reward: nil | number()}.nilon a failed attempt.- Metadata:
%{attempt: non_neg_integer(), kept?: boolean(), failed?: boolean(), fed_back?: boolean()}merged over the resolvedDsxir.Settings:metadataframe (so tenant metadata propagates).
[:dsxir, :predictor, :best_of_n | :refine, :stop]
- Measurements:
%{best_reward: number()}. - Metadata: the resolved
Dsxir.Settings:metadataframe.
Summary
Functions
Event name for [:dsxir, :adapter, :fallback].
Event name for [:dsxir, :adapter, :format].
Event name for [:dsxir, :adapter, :parse].
Thin wrapper over :telemetry.execute/3.
Event name for [:dsxir, :predictor, :ensemble, :member] (one per ensembled program).
Event name for [:dsxir, :predictor, :ensemble, :stop] (one per Ensemble.run/3).
Event name for [:dsxir, :evaluate, :item].
Event name for [:dsxir, :evaluate, :stop].
Event name for [:dsxir, :lm, :embed, :stop] (one per successful Dsxir.LM.embed/2 dispatch).
Event name for [:dsxir, :miprov2, :proposer] (point-in-time proposer outcome).
Event name for [:dsxir, :miprov2, :rerank] (point-in-time full-eval rerank).
Event name for [:dsxir, :optimizer, :item_error].
Event name for [:dsxir, :optimizer, :start].
Event name for [:dsxir, :optimizer, :stop].
Event name for [:dsxir, :optimizer, :trial].
Event name for [:dsxir, :predictor, :code_exec, :attempt].
Event name for [:dsxir, :predictor, :exception].
Event name for [:dsxir, :predictor, :start].
Event name for [:dsxir, :predictor, :stop].
Event name for the sampling-predictor :attempt event, parameterized by :best_of_n or :refine.
Event name for the sampling-predictor :stop event, parameterized by :best_of_n or :refine.
Types
@type event() :: [atom(), ...]
Functions
@spec adapter_fallback() :: event()
Event name for [:dsxir, :adapter, :fallback].
@spec adapter_format() :: event()
Event name for [:dsxir, :adapter, :format].
@spec adapter_parse() :: event()
Event name for [:dsxir, :adapter, :parse].
Thin wrapper over :telemetry.execute/3.
@spec ensemble_member() :: event()
Event name for [:dsxir, :predictor, :ensemble, :member] (one per ensembled program).
@spec ensemble_stop() :: event()
Event name for [:dsxir, :predictor, :ensemble, :stop] (one per Ensemble.run/3).
@spec evaluate_item() :: event()
Event name for [:dsxir, :evaluate, :item].
@spec evaluate_stop() :: event()
Event name for [:dsxir, :evaluate, :stop].
@spec lm_embed_stop() :: event()
Event name for [:dsxir, :lm, :embed, :stop] (one per successful Dsxir.LM.embed/2 dispatch).
@spec miprov2_proposer() :: event()
Event name for [:dsxir, :miprov2, :proposer] (point-in-time proposer outcome).
@spec miprov2_rerank() :: event()
Event name for [:dsxir, :miprov2, :rerank] (point-in-time full-eval rerank).
@spec optimizer_item_error() :: event()
Event name for [:dsxir, :optimizer, :item_error].
@spec optimizer_start() :: event()
Event name for [:dsxir, :optimizer, :start].
@spec optimizer_stop() :: event()
Event name for [:dsxir, :optimizer, :stop].
@spec optimizer_trial() :: event()
Event name for [:dsxir, :optimizer, :trial].
@spec predictor_code_exec_attempt() :: event()
Event name for [:dsxir, :predictor, :code_exec, :attempt].
@spec predictor_exception() :: event()
Event name for [:dsxir, :predictor, :exception].
@spec predictor_start() :: event()
Event name for [:dsxir, :predictor, :start].
@spec predictor_stop() :: event()
Event name for [:dsxir, :predictor, :stop].
@spec sampling_attempt(:best_of_n | :refine) :: event()
Event name for the sampling-predictor :attempt event, parameterized by :best_of_n or :refine.
@spec sampling_stop(:best_of_n | :refine) :: event()
Event name for the sampling-predictor :stop event, parameterized by :best_of_n or :refine.