Telemetry

What Crosswake Telemetry Is

Crosswake.Telemetry is the canonical public API for the :telemetry events Crosswake emits across its companion, doctor, threadline, sigra, and chimeway subsystems.

Call Crosswake.Telemetry.events/0 at runtime to retrieve a self-describing catalog of every event Crosswake declares. Each entry carries the event name prefix, a tier (:active for events that fire today, :reserved for events declared but not yet emitted), a description, and the exact measurement and metadata keys the event carries. This catalog is the source of truth for host observability pipelines, dashboards, and contract tests.

Telemetry in Crosswake is diagnostic-only. The library emits standard :telemetry events and exposes a typed, low-cardinality, PII-free catalog. It coexists with any host-side observability pipeline without requiring changes to existing instrumentation. No new dependencies are introduced — only :telemetry, which is already a project dependency.

What Crosswake Telemetry Is NOT

These are non-goals by design, not deferred features.

• Not an APM / observability platform. Crosswake.Telemetry emits events and catalogs them. It does not collect, ingest, sample, or aggregate telemetry. Host teams choose their own pipeline.
• Not a distributed tracing framework. Crosswake does not create or manage trace context, spans in the OTel sense, or baggage. If your host uses OpenTelemetry, attach a host-owned OTel handler to bridge the :telemetry events into your OTel pipeline.
• Not a generic event bus. The event catalog is typed, low-cardinality, and scoped to Crosswake subsystem activity. It is not an open subscription API for arbitrary host events.
• Not a source of PII. All metadata passes through a forbidden-key denylist before emission. PII-bearing fields are silently dropped. See Security and PII.

Semver Contract

Telemetry events are public API.

• Additions (new event prefixes, new measurement keys, new metadata keys) are non-breaking minor releases.
• Removals or renames of existing event prefixes, measurement keys, or metadata keys are breaking changes requiring a semver major bump.

Host code that attaches handlers to Crosswake events may depend on this stability guarantee.

Events

The table below lists every :active event from Crosswake.Telemetry.events/0 — events that fire today in production code. Event names follow the Keathley span convention: a prefix atom list expanded to :start, :stop, and :exception suffixes.

For span-based events (emitted via :telemetry.span/3), the :start event carries system_time in measurements and the :stop/:exception events carry duration. For the four companion spans, companion_id and route_id are passed as the span context map (the second argument to :telemetry.span/3), which :telemetry places in metadata, not measurements — this is the Keathley span convention.

Stop metadata is a superset of start metadata for span-based events. See the threadline :exception caveat below.

Companion: dependency_check

Emitted when RouteGate checks a companion's optional dependency presence.

Event names:

• `[:crosswake, :companion, :dependency_check, :start]`
• `[:crosswake, :companion, :dependency_check, :stop]`
• `[:crosswake, :companion, :dependency_check, :exception]`

Companion: kill_switch

Emitted when RouteGate evaluates a companion's kill switch. Short-circuits ahead of route_gate evaluation.

Event names:

• `[:crosswake, :companion, :kill_switch, :start]`
• `[:crosswake, :companion, :kill_switch, :stop]`
• `[:crosswake, :companion, :kill_switch, :exception]`

Companion: route_gate

Emitted when RouteGate evaluates a companion's route policy.

Event names:

• `[:crosswake, :companion, :route_gate, :start]`
• `[:crosswake, :companion, :route_gate, :stop]`
• `[:crosswake, :companion, :route_gate, :exception]`

Companion: validate_dependency

Emitted when Doctor runs validate_dependency/0 for each registered companion. The :stop event carries an additional result metadata key.

Event names:

• `[:crosswake, :companion, :validate_dependency, :start]`
• `[:crosswake, :companion, :validate_dependency, :stop]`
• `[:crosswake, :companion, :validate_dependency, :exception]`

Threadline: request

Emitted by Crosswake.Plug.Threadline for each incoming HTTP request correlation span. Unlike the companion spans above, this event is emitted via discrete :telemetry.execute calls (not :telemetry.span/3), so measurements and metadata are set explicitly per suffix.

Threadline :exception metadata caveat. The :exception event is emitted when a plug exception occurs before the correlation context is fully established. In this case, kind and reason are passed as metadata, but they are not in the threadline metadata allowlist (thread_id, correlation_id, route_id, source) and are dropped by the PII guard. The exception event carries timing measurements (duration) but the metadata map may be effectively empty. Do not assert that thread_id is present in a threadline :exception event — the stop ⊇ start metadata guarantee holds for :stop but not for :exception in the threadline case.

Event names:

• `[:crosswake, :threadline, :request, :start]`
• `[:crosswake, :threadline, :request, :stop]`
• `[:crosswake, :threadline, :request, :exception]`

Reserved Events

Sigra and Chimeway each declare event catalogs (via event_names/0) that are not yet emitted from production code. These appear in Crosswake.Telemetry.events/0 with tier: :reserved.

Reserved events are:

• Included in events/0 so adopters can see what is planned and attach handlers speculatively.
• Excluded from the declared⇒emitted contract test (they do not fire yet).
• Included in the emitted⇒declared contract test (if code accidentally starts firing them, it will be caught).

Sigra declares 14 reserved events; Chimeway declares 10. Their event names follow the `[:crosswake, :sigra, ...]` and `[:crosswake, :chimeway, ...]` prefixes respectively. Filter by tier: :reserved in events/0 to enumerate them.

Crosswake.Offline.Telemetry is a metadata-contract module only; it has no event catalog and contributes no events to events/0.

Attaching the Default Logger

Core never attaches a telemetry handler automatically. Attachment is always the host's explicit decision.

To attach the default structured logger to all :active Crosswake events, call Crosswake.Telemetry.attach_default_logger/1 from your application's supervision tree or Application.start/2:

# In your Application.start/2 or a startup module:
:ok = Crosswake.Telemetry.attach_default_logger()

The function accepts a log level atom or a keyword list:

# Attach with a specific log level
Crosswake.Telemetry.attach_default_logger(:debug)

# Attach with keyword options
Crosswake.Telemetry.attach_default_logger(level: :info, encode: false)

Options:

The handler is registered under the id "crosswake-default-logger". Log messages are prefixed with [crosswake].

To remove the handler:

Crosswake.Telemetry.detach_default_logger()

Failure Modes

No companions configured. events/0 returns core and in-tree events only. It never raises. This is a valid state, not a misconfiguration.

Double-attach. Calling attach_default_logger/1 when a handler with id "crosswake-default-logger" is already registered returns {:error, :already_exists}. This is the standard :telemetry behavior — Crosswake does not add a custom guard. Check the return value and decide whether to treat it as an error in your startup logic.

Companion module not loaded. events/0 probes companions via function_exported?/3. If a companion module is not yet loaded when events/0 is called, its events are excluded silently. Call events/0 after companions are started in your supervision tree.

Handler raises. If an attached telemetry handler raises an exception, :telemetry catches it and detaches the handler. The [crosswake] default logger handles this per standard :telemetry semantics.

Security and PII

Crosswake telemetry is PII-free by construction.

Forbidden metadata keys. All metadata passes through a forbidden-key denylist before emission or logging. The denylist is the union of forbidden_metadata_keys/0 from all subsystem telemetry modules:

• Crosswake.Threadline.Telemetry.forbidden_metadata_keys/0 — 20 keys including access_token, actor_id, actor_ref, authorization_code, device_id, email, id_token, ip, refresh_token, session_ref, and others.
• Crosswake.Companions.Sigra.Telemetry.forbidden_metadata_keys/0 — additional keys from the Sigra subsystem.
• Crosswake.Companions.Chimeway.Telemetry.forbidden_metadata_keys/0 — additional keys from the Chimeway subsystem.

Forbidden keys are silently dropped before metadata reaches any telemetry handler. They are never logged, even with the default logger attached.

High-cardinality values. A safe_value?/1 guard in each subsystem telemetry module rejects metadata values longer than 128 characters. This prevents high-cardinality strings (e.g., full request bodies) from leaking into telemetry metadata.

Host responsibility. Companions and host code that add metadata to events must not include PII. The denylist covers known PII field names, but domain-specific PII with non-standard field names must be excluded by the host.

Do not add identity tokens, user emails, IP addresses, or other PII to telemetry metadata — even in development. Telemetry events are forwarded to external systems (APM, log aggregators, dashboards) where PII retention policies apply.

Testing

Use :telemetry_test.attach_event_handlers/2 in host test suites to assert that expected events fire:

defmodule MyApp.TelemetryTest do
  use ExUnit.Case, async: false

  test "companion route gate emits telemetry" do
    event_names = [
      [:crosswake, :companion, :route_gate, :start],
      [:crosswake, :companion, :route_gate, :stop]
    ]

    ref = :telemetry_test.attach_event_handlers(self(), event_names)
    on_exit(fn -> :telemetry.detach(ref) end)

    # Drive the code path that triggers RouteGate evaluation
    # ...

    assert_received {[:crosswake, :companion, :route_gate, :start], ^ref, measurements, _metadata}
    assert Map.has_key?(measurements, :system_time)

    assert_received {[:crosswake, :companion, :route_gate, :stop], ^ref, measurements, _metadata}
    assert Map.has_key?(measurements, :duration)
  end
end

attach_event_handlers/2 returns a reference used for ^ref pattern matching, ensuring received messages originated from this handler. Always detach in on_exit to avoid handler leakage between tests.

Use async: false for any test that modifies Application.get_env(:crosswake, :companions, ...) — the companions config key is shared globally.

To derive the full event list from the catalog rather than hardcoding it:

active_event_names =
  Crosswake.Telemetry.events()
  |> Enum.filter(fn e -> e.tier == :active end)
  |> Enum.flat_map(fn %{event: prefix} ->
    [prefix ++ [:start], prefix ++ [:stop], prefix ++ [:exception]]
  end)

ref = :telemetry_test.attach_event_handlers(self(), active_event_names)

Related

• Companion Contract — the behaviour contract for companions, including the optional telemetry_events/0 callback.
• Threadline — the HTTP correlation thread, PII-free metadata, and the ledger schema.