View Source Overview
opentelemetry_xray
OpenTelemetry AWS X-Ray support for Erlang/Elixir.
This library includes the following modules:
An id generator that creates X-Ray-compatible
trace_id
andspan_id
. It implements theotel_id_generator
protocol in the Erlang SDK.A propagator that reads and writes AWS X-Ray trace context headers. It implements the
otel_propagator_text_map
protocol in the Erlang SDK.
It assumes that you are using the AWS Distro for OpenTelemetry Collector, a version of the OpenTelemetry Collector which has support for AWS services such as X-Ray. This collector accepts standard OpenTelemetry traces, converts them to X-Ray format, and sends them to AWS. You can run the collector as a sidecar container in an ECS task or as a daemon on an EC2 instance.
In AWS X-Ray, the trace_id
is a 128-bit value. The first 32 bits are a Unix
time_t
and the rest are a 96-bit random number. If you use the default
trace_id
, then X-Ray will reject your traces. This library generates ids that
are compatible with X-Ray.
When your app app is downstream from another app or the AWS load balancer, the
upstream app creates the trace for the current request and sends it in the
X-Amzn-Trace-Id" HTTP header. The header includes the trace id and optional information about the parent span and sampling. This library includes a propagator which reads the trace id from this header and uses it within your app. It can then pass the same trace id to downstream apps via the header. NOTE: By default Amazon samples relatively few traces. If you want to ensure that your traces are sampled, make sure that you turn on sampling in your app. A common approach is to turn on sampling for all traces that have errors, and some percentage of normal traces. Links: * Propagators in general: https://opentelemetry.io/docs/specs/otel/context/api-propagators/ * Erlang SDK propagation: https://opentelemetry.io/docs/instrumentation/erlang/propagation/ * Erlang SDK id generator: https://github.com/open-telemetry/opentelemetry-erlang/blob/main/apps/opentelemetry/src/otel_id_generator.erl * X-Ray tracing header: https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html#xray-concepts-tracingheader * X-Ray sampling: https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html#xray-concepts-sampling * X-Ray configuration: https://aws-otel.github.io/docs/getting-started/x-ray#configuring-the-aws-x-ray-exporter * AWS CloudWatch Log Group resource: https://github.com/aws/aws-xray-sdk-python/issues/188 * OpenTelemetry resources: https://opentelemetry.io/docs/instrumentation/erlang/resources/ * OpenTelemetry getting started: https://opentelemetry.io/docs/instrumentation/erlang/getting-started/ * OpenTelemetry intro: https://davelucia.com/blog/observing-elixir-with-lightstep * Addding Erlang logger filters to new Elixir logger: https://write.as/yuriploc/elixir-logger-and-erlang-filters ## Installation Erlang: Add
opentelemetry_xrayto the list of dependencies in
rebar.config: ```erlang {deps, [opentelemetry_xray]}. ``` Elixir: Add
opentelemetry_xrayto the list of dependencies in
mix.exs: ```elixir def deps do [ {:opentelemetry_xray, "~> 0.7"}, ] end ``` ## Configuration Erlang: In
sys.config: ```erlang {opentelemetry, { id_generator, [opentelemetry_xray_id_generator], propagators: [opentelemetry_xray_propagator, baggage] }} ``` Elixir: In
config/prod.exs, configure
opentelemetry` to use this library:
elixir config :opentelemetry, id_generator: :opentelemetry_xray_id_generator, propagators: [:opentelemetry_xray_propagator, :baggage]
Add resource attributes to the span to connect it to log messages:
shell export OTEL_RESOURCE_ATTRIBUTES="aws.log.group.names=$AWS_LOG_GROUP"
See phoenix_container_example
for a complete Elixir Phoenix app that uses this library.