How to Collect Telemetry and Metrics

Copy Markdown View Source

rpc_load_balancer instruments RPC calls and node selection with :telemetry, and ships Telemetry.Metrics definitions you can hand to any reporter (Prometheus, StatsD, LiveDashboard).

Register the bundled metrics

RpcLoadBalancer.Metrics.metrics/0 returns a list of Telemetry.Metrics structs. Add them to your reporter — with PrometheusTelemetry:

# in your application.ex
children = [
  {PrometheusTelemetry,
   exporter: [enabled?: prod?()],
   metrics: [
     RpcLoadBalancer.Metrics.metrics(),
     PrometheusTelemetry.Metrics.VM.metrics()
   ]}
]

Or with TelemetryMetricsPrometheus / LiveDashboard:

{TelemetryMetricsPrometheus, metrics: RpcLoadBalancer.Metrics.metrics()}

That exposes:

SeriesTypeTagsMeaning
rpc_load_balancer.rpc.request.start.countcounternode module function type load_balancerRPCs started
rpc_load_balancer.rpc.request.stop.countcounter... + statusRPCs completed
rpc_load_balancer.rpc.duration.millisecondsdistribution... + statusRPC latency; buckets from 0.1 ms to 60 s
rpc_load_balancer.node.selected.countcounteralgorithm load_balancer nodeSelections per target — reveals skew
rpc_load_balancer.node.selected.empty.countcounteralgorithm load_balancerSelection attempted against an empty pool
rpc_load_balancer.node.pool_sizedistributionalgorithm load_balancerMember count observed at selection time

:module is the inspect/1'd module name (a string), and :load_balancer is nil for direct (non-balanced) calls — both are bounded, so cardinality stays sane. :node is one label per cluster member.

Raw events

If you want to attach your own handlers, these are the events.

RPC span — [:rpc_load_balancer, :rpc, ...]

Every RpcLoadBalancer.call/5 and cast/5 (direct or load-balanced, including through use RpcLoadBalancer modules and the random-node helpers) is wrapped in a :telemetry.span/3:

EventMeasurementsNotes
[:rpc_load_balancer, :rpc, :start]system_time
[:rpc_load_balancer, :rpc, :stop]duration (native units)adds :status to metadata
[:rpc_load_balancer, :rpc, :exception]durationremote :erpc failures come back as {:error, ...} and land in :stop; this fires when the wrapped work raises locally — e.g. apply/3 under CallDirect / call_directly?

Metadata on all three:

KeyValue
:type:call or :cast
:nodethe node argument passed to call/5/cast/5 — for load-balanced calls this is the caller's argument (usually node()), not the selected target. The random-node helpers call call/5 internally with the node they picked, so there :node is the real target and :load_balancer is nil
:moduleinspect(module)
:functionfunction name atom
:load_balancerbalancer name / module, or nil
:status (stop only):ok, the ErrorMessage code (:request_timeout, :service_unavailable, :bad_request), or :error

For load-balanced calls the span covers the whole routed operation, including any no-route retry sleeps.

Node selection — [:rpc_load_balancer, :node_selected]

Emitted after every successful choose_from_nodes/3 (custom algorithms included):

  • Measurements: %{count: 1, members_count: pool_size}
  • Metadata: %{algorithm: module, load_balancer: name, node: selected_node}

[:rpc_load_balancer, :node_selected, :empty] fires when an algorithm is invoked with an empty member list (measurements %{count: 1}, metadata without :node). Through the public API this can't happen — select_node/2 returns an error before selection — but the event exists so direct users of SelectionAlgorithm.choose_from_nodes/4 still see the miss.

Selection events are not emitted when the balancer runs CallDirect or call_directly? is set — nothing is selected.

LeastCpu poller — [:rpc_load_balancer, :least_cpu, :poll, ...]

:start/:stop/:exception span per poll cycle plus :remote_error per failed remote fetch. Not part of Metrics.metrics/0. See Route by CPU Load with LeastCpu.

Attach a handler directly

:telemetry.attach(
  "log-slow-rpcs",
  [:rpc_load_balancer, :rpc, :stop],
  fn _event, %{duration: duration}, metadata, _config ->
    ms = System.convert_time_unit(duration, :native, :millisecond)

    if ms > 1_000 do
      Logger.warning("slow rpc #{metadata.module}.#{metadata.function} #{ms}ms status=#{metadata.status}")
    end
  end,
  nil
)

What is deliberately not instrumented

The hot path inside selection algorithms reads caches through raw adapter calls (:counters.get/2, :ets.lookup/2, :persistent_term.get/2) rather than elixir_cache's telemetry-wrapped get/1. You get one node_selected event per selection, not one per cache read — see bench/README.md for why.