ExAgent. Observability. BoundedProcessor
(ExAgent v2.1.0)
View Source
An application-owned, bounded native OpenTelemetry span processor.
config :opentelemetry,
processors: [
{ExAgent.Observability.BoundedProcessor,
%{name: :agent_export,
exporter: {:opentelemetry_exporter, %{}},
max_queue_size: 2048,
max_export_batch_size: 256,
scheduled_delay_ms: 1000,
exporting_timeout_ms: 2000}}
]The application must include the optional OpenTelemetry SDK when compiling
this adapter. This module neither starts nor replaces a tracer provider. The
SDK starts it through the public :otel_span_processor extension. Configure
:processors, not both :processors and :span_processor. SDK bsp_* options
are not automatically applied to custom processors; pass options here.
Admission uses a finite ETS slot table, without sending a message per span.
Slots stay occupied until export completes or fails, including the one batch
in flight. At most max_queue_size distinct admitted spans are retained. A
bounded scan may conservatively drop during concurrent slot turnover. Batch
copies, native serialization and exporter allocations are additional memory;
this is a count bound, not a byte limit. Configure SDK attribute/event limits
and bound/redact content before creating spans. Active SDK spans are separate.
One guarded exporter process owns initialization, export and shutdown. Exports and initialization have a monotonic deadline, enforced outside that process. A failed batch is discarded, never retried. After an export worker dies it is replaced once; failed/timed-out initialization disables admission until the processor is restarted. An independent owner monitor kills our worker even when an exporter traps exit signals or blocks inside a callback. Killing a worker does not guarantee remote rollback or cleanup of processes created independently by a custom exporter.
max_exporter_restarts caps automatic worker replacements per processor
instance. It accepts a non-negative integer, or :infinity (the existing
default). Zero allows the initial worker only. After exhaustion, admission is
disabled, remaining queued spans are discarded as dropped_unavailable, and
the processor stays alive with status :unavailable until explicitly restarted.
exporter_restarts counts replacements, excluding initial startup;
restart_limit_reached counts exhaustion. Completed error callbacks do not
recreate a worker or spend this budget. It resets with the processor instance,
not across SDK/application/VM restarts. This limits recreation of exporter-owned
resources; it does not reclaim a native HTTP profile/socket or retained atoms.
force_flush/1 coalesces an asynchronous request to start/drain batches. Its
:ok is not an exporter or remote-delivery acknowledgement. Use an exporter
barrier when testing. shutdown/1 closes admission and discards remaining
spans; it does not flush. Exporter shutdown is bounded by
shutdown_timeout_ms (default 1000, maximum 4000). A busy exporter is killed
rather than concurrently invoking its shutdown callback. Abrupt VM/process
loss can lose spans and counters. In-flight admission racing shutdown is
best-effort and may not appear in its final counter snapshot.
stats/1 returns cumulative, generation-local counters and a point-in-time
queue snapshot. Counters are operational diagnostics, not a delivery ledger.
Every scheduled interval an isolated, single observer emits the same numeric
measurements on [:exagent, :observability, :processor], with only a fixed
:status metadata value. Slow telemetry handlers cannot block admission or
export deadlines; they can delay these periodic notifications. No raw errors,
exporter state, configuration, names or spans are emitted by this processor.
A native/custom exporter may have its own logging policy.
Bootstrap configuration must not contain secrets. The processor options,
including exporter: {module, opts}, are non-secret bootstrap descriptors:
SDK/OTP supervisors retain their original start arguments and may log them on
startup or restart, before this module's status/error projection can act. Use
exporter: {:opentelemetry_exporter, %{}} and configure credentials through the
exporter's application environment or OS environment. Custom exporters should
likewise resolve credentials inside init/1, rather than accept credentials
in their supervised start arguments. No general validator can identify all
arbitrary secrets, and this processor does not intercept application logging.
The SDK short-circuits later processors when an earlier on_end drops. When
composing independent destinations, the application must choose/test their
ordering. This processor receives all scopes routed through its provider.
resource may be supplied explicitly; otherwise the SDK resource detector is
used. Choose stable instance names; name registration is node-qualified and
never creates atoms from runtime names. Old tracer configurations resolve the
current instance after a restart, rather than retaining a dead PID/table.
shutdown/1 stops one instance, not the SDK child specification: a permanent
SDK child may restart. To uninstall a route, the app must change its provider's
supervision/configuration rather than rely on this helper.
Summary
Functions
Returns a specification to start this module under a supervisor.
Request asynchronous draining; success is not an export acknowledgement.
Callback implementation for GenServer.format_status/1.
Callback implementation for GenServer.init/1.
Discard pending spans and stop this instance, returning its final snapshot.
Start through the SDK processor extension; returns a stable callback config.
Read generation-local counters and queue gauges without calling the processor.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.
Request asynchronous draining; success is not an export acknowledgement.
Callback implementation for GenServer.format_status/1.
Callback implementation for GenServer.init/1.
Discard pending spans and stop this instance, returning its final snapshot.
Start through the SDK processor extension; returns a stable callback config.
Read generation-local counters and queue gauges without calling the processor.