Observability with Telemetry and OpenTelemetry

When you run an LLM chain in production, you want to answer questions like: How long did that request take? How many tokens did it burn? Which tool call failed? Where is the latency — the model, or my own tool code? LangChain answers these through two layers you can adopt independently:

1. LangChain.Telemetry — vendor-neutral :telemetry events emitted for every LLM call, chain execution, and tool call. Attach your own handlers, feed them into Telemetry.Metrics, PromEx, Logger, or anything else. No extra dependencies.
2. LangChain.OpenTelemetry — an optional, opt-in integration that translates those events into OpenTelemetry spans and metrics following a subset of the GenAI Semantic Conventions (v1.40+). This is what you want for distributed tracing backends like Langfuse, Honeycomb, Grafana Tempo, Jaeger, or any OTLP collector.

Layer 2 is built on layer 1, so everything below applies whether you consume the raw events yourself or let the OpenTelemetry handler do it for you.

The telemetry layer

LangChain.Telemetry emits standard :telemetry events using the naming convention [:langchain, component, operation, stage]. The core lifecycle events are:

Each :start/:stop/:exception triple shares a :call_id (a UUID) so you can correlate them. :stop events carry a duration measurement (in native time units); :exception events carry one too, so failed operations remain visible to latency metrics.

Key metadata fields

• :provider — the LLM provider ("openai", "anthropic", "google", …), available on LLM call events.
• :token_usage — a %LangChain.TokenUsage{} struct on LLM call :stop events (when the model reports usage) and on chain :stop events (aggregated across all assistant messages in the run, so multi-turn/tool-calling chains don't lose earlier turns' tokens).
• :custom_context — your LLMChain.custom_context, surfaced on chain and tool events.
• :last_message — the final assembled %Message{} on chain :stop events.

Attaching your own handler

:telemetry.attach(
  "my-llm-logger",
  [:langchain, :llm, :call, :stop],
  fn _event, measurements, metadata, _config ->
    ms = System.convert_time_unit(measurements.duration, :native, :millisecond)

    require Logger
    Logger.info(
      "LLM #{metadata.provider}/#{metadata.model} took #{ms}ms, " <>
        "tokens: #{inspect(metadata[:token_usage])}"
    )
  end,
  nil
)

A privacy note

Message content is intentionally excluded from the lifecycle events (:start / :stop / :exception) to avoid unconditionally exposing user or PII data to every handler. Content is only available through the purpose-specific [:langchain, :llm, :prompt] and [:langchain, :llm, :response] events — subscribing to those is an explicit opt-in.

See LangChain.Telemetry for the full list of events and the exact metadata shape of each.

The OpenTelemetry layer

Installation

The OpenTelemetry integration ships in LangChain but its dependencies are optional — nothing is pulled in and no code is compiled unless you add them. Add these to your application's mix.exs:

def deps do
  [
    {:langchain, "~> 0.8"},
    # OpenTelemetry — required only if you want the OTel integration
    {:opentelemetry_api, "~> 1.4"},
    {:opentelemetry, "~> 1.5"},
    {:opentelemetry_exporter, "~> 1.8"}
  ]
end

Setup

Call setup/1 once during application startup — for example in your Application.start/2:

defmodule MyApp.Application do
  use Application

  def start(_type, _args) do
    LangChain.OpenTelemetry.setup()

    children = [
      # ...
    ]

    Supervisor.start_link(children, strategy: :one_for_one, name: MyApp.Supervisor)
  end
end

That attaches the span and metrics handlers to LangChain's telemetry events. From then on, every chain run, LLM call, and tool call produces OTel spans automatically. To detach later, call LangChain.OpenTelemetry.teardown/0.

What spans you get

Spans follow a subset of the GenAI Semantic Conventions (see LangChain.OpenTelemetry.Attributes for the attributes emitted) and nest automatically for synchronous work (they share the process dictionary):

invoke_agent llm_chain              (:internal)   gen_ai.operation.name = invoke_agent
  └─ chat gpt-4o                     (:client)     gen_ai.operation.name = chat
       └─ execute_tool get_weather   (:internal)   gen_ai.operation.name = execute_tool

Attributes recorded on the spans include (a subset, always present when available):

• gen_ai.operation.name, gen_ai.provider.name
• gen_ai.output.type — "json" when the model requests structured output (json_response / json_schema / a JSON response_format), otherwise "text"
• gen_ai.request.model, gen_ai.response.model
• server.address, server.port — derived from the model's request endpoint
• Request parameters, when the model sets them: gen_ai.request.temperature, gen_ai.request.max_tokens, gen_ai.request.top_p, gen_ai.request.top_k, gen_ai.request.frequency_penalty, gen_ai.request.presence_penalty, gen_ai.request.seed, gen_ai.request.choice.count, gen_ai.request.stream, gen_ai.request.stop_sequences, gen_ai.request.reasoning.level
• gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and — best-effort, for providers that report them — gen_ai.usage.cache_read.input_tokens, gen_ai.usage.cache_creation.input_tokens, gen_ai.usage.reasoning.output_tokens
• gen_ai.response.finish_reasons — best-effort ("stop" / "length" / "tool_calls"), reconstructed from the normalized message status
• gen_ai.response.time_to_first_token — for streaming calls, seconds from the request start to the first streamed chunk. A gen_ai.first_token span event is also recorded at that moment so backends can place it on the trace timeline.
• gen_ai.tool.name, gen_ai.tool.call.id, gen_ai.tool.type, gen_ai.tool.description
• gen_ai.agent.name (the chain type), gen_ai.conversation.id (see below)

Provider names are normalized to the OTel registry — e.g. "google" becomes gcp.gemini, "vertex_ai" becomes gcp.vertex_ai, "xai" becomes x_ai, and "mistralai" becomes mistral_ai. Unknown providers pass through unchanged. See LangChain.OpenTelemetry.ProviderMapping.

Grouping traces into conversations

LangChain has no first-class conversation id, so gen_ai.conversation.id (which backends use to group a multi-turn session's traces) is read from your chain's custom_context. Set a :conversation_id key — or reuse the :langfuse_session_id you may already set — and it lands on the root chain span:

chain
|> Map.put(:custom_context, %{conversation_id: session_id})

Identifying the agent

Two more attributes come from custom_context when you set them:

The gen_ai.agent.name fallback is the same value for every LLMChain, so it can't group anything on its own. Frameworks built on LangChain generally already track a name and id per agent; passing them through is what makes the attribute useful.

Attaching your own context to spans

Traces get much more useful when spans carry your domain context — which tenant, which user, which feature. Set a flat map under the reserved :otel_attributes key in custom_context:

chain
|> LLMChain.update_custom_context(%{
  otel_attributes: %{
    "user.id" => current_user.id,
    "organization.id" => org.id,
    "myapp.feature" => "support_chat",
    "myapp.plan" => org.plan
  }
})

Those attributes land on every span the chain produces — the invoke_agent span, each chat span, and each execute_tool span — so you can filter and group at the span level, not just the trace level. gen_ai.conversation.id is inherited the same way.

Only the reserved key is exported, never the rest of custom_context. That map routinely holds structs, PIDs, and whole conversation states; copying it wholesale would produce enormous spans and export data you never intended to send.

A few things worth knowing:

• Namespace your keys (myapp.*) per the semantic conventions. A gen_ai.* key is honoured deliberately, so you can fill a convention slot LangChain doesn't populate itself.
• Types are preserved. Strings, numbers, booleans, and homogeneous lists of those stay native, so numeric attributes remain filterable as numbers. Anything else is JSON-encoded. nil values are dropped.
• Your values never mask real ones. On a span that derives an attribute itself, the derived value wins — an inherited gen_ai.request.model can't overwrite the actual model on a chat span. The one exception is the chain span, where an :otel_attributes entry is taken as a deliberate override.
• Keep cardinality sane. Tenant, user, and conversation ids are fine because backends index on them. Free-text values (titles, queries) are not.

Inheritance rides the OpenTelemetry context, so it follows the trace across process boundaries — including async: true tools running in their own Task — but is never serialized onto outbound HTTP requests the way OpenTelemetry baggage is. Set inherit_attributes: false in setup/1 to keep attributes on the chain span only.

Enriching a span mid-run

For values you only learn once work is underway, LangChain.OpenTelemetry.Enrich sets attributes on whichever span is currently open:

# inside a tool function — lands on that tool's `execute_tool` span
LangChain.OpenTelemetry.Enrich.set_current_span_attributes(%{
  "myapp.records_matched" => length(rows),
  "myapp.cache" => "hit"
})

To seed context before any chain exists — in a Plug, a LiveView mount/3, an Oban worker — use put_inherited_attributes/1 instead, and every LangChain span opened later in that process inherits it:

LangChain.OpenTelemetry.Enrich.put_inherited_attributes(%{
  "organization.id" => conn.assigns.current_org.id
})

Both functions are safe to call unconditionally: unlike the rest of the OpenTelemetry integration, Enrich compiles to no-ops when :opentelemetry_api isn't available, so your code doesn't need its own guards or an OpenTelemetry dependency.

Errors

When an operation raises, its span is closed with an error status, the exception is recorded on the span, and an error.type attribute (the exception's module name) is set so backends can group and filter by error kind.

Configuration options

All options are passed to setup/1 and default to the safest choice. See LangChain.OpenTelemetry.Config.

# Trace latency and token usage, but never record message content (default):
LangChain.OpenTelemetry.setup()

# Full-fidelity tracing, including message content — only where PII is acceptable:
LangChain.OpenTelemetry.setup(
  capture_input_messages: true,
  capture_output_messages: true,
  capture_tool_arguments: true,
  capture_tool_results: true
)

Message and tool-content capture is off by default precisely because those attributes can carry sensitive data. Turn them on deliberately, and only where your tracing backend and data-retention policy allow it.

Metrics require a consumer

Example wiring with Telemetry.Metrics. Name the metrics with the canonical GenAI semantic-convention names (gen_ai.client.operation.duration and gen_ai.client.token.usage) so they line up with other OTel GenAI instrumentation, and configure the spec's recommended histogram buckets:

defmodule MyApp.Telemetry do
  import Telemetry.Metrics

  # Recommended explicit bucket boundaries from the GenAI semantic conventions.
  @duration_buckets [0.01, 0.02, 0.04, 0.08, 0.16, 0.32, 0.64, 1.28, 2.56, 5.12,
                     10.24, 20.48, 40.96, 81.92]
  @token_buckets [1, 4, 16, 64, 256, 1024, 4096, 16_384, 65_536, 262_144,
                  1_048_576, 4_194_304, 16_777_216, 67_108_864]

  def metrics do
    [
      distribution("gen_ai.client.operation.duration",
        event_name: [:langchain, :otel, :operation, :duration],
        measurement: :duration_s,
        unit: :second,
        reporter_options: [buckets: @duration_buckets],
        tags: [:"gen_ai.operation.name", :"gen_ai.provider.name", :"gen_ai.request.model"]
      ),
      distribution("gen_ai.client.token.usage",
        event_name: [:langchain, :otel, :token, :usage],
        measurement: :tokens,
        reporter_options: [buckets: @token_buckets],
        tags: [:"gen_ai.operation.name", :"gen_ai.provider.name", :"gen_ai.token.type"]
      )
    ]
  end
end

See LangChain.OpenTelemetry.MetricsHandler for the exact event shapes.

Suppressing traces for utility chains

Some chains — a translation pass, a title generator, a routing decision — are implementation details you may not want cluttering your traces. Wrap them in without_tracing/1 and no spans are exported — and, when metrics are enabled, no duration/token metric events are emitted — for anything inside:

LangChain.OpenTelemetry.without_tracing(fn ->
  {:ok, updated_chain} = LLMChain.run(title_chain)
  updated_chain
end)

Suppression is scoped to the calling process. Work that runs in a separate process spawned inside the block (e.g. an async: true tool) does not inherit it, the same way OTel span context does not cross a process boundary.

Async tools

Tools declared with async: true execute in a separate Task process. The OpenTelemetry context lives in the parent process's dictionary and is not inherited by a spawned process, so an async tool's span would otherwise become its own root span instead of a child of the chain span.

For tools run by LLMChain's built-in tool executor this is now handled automatically: the chain captures the current OpenTelemetry context before spawning each async tool Task and re-attaches it inside, so async tool spans nest under the chain span with no extra work. (The propagation is a no-op unless you have the OpenTelemetry dependencies installed.)

If you spawn your own processes that run LangChain operations, the same rule applies to you — the context doesn't cross the process boundary by itself. Capture OpenTelemetry.Ctx.get_current/0 before spawning and OpenTelemetry.Ctx.attach/1 inside the spawned process. The :on_tool_pre_execution callback (which fires inside the spawned process) is one convenient place to do this.

Langfuse integration

Langfuse ingests OpenTelemetry traces via its OTLP-compatible endpoint. Point the OTel exporter at your Langfuse instance:

# config/runtime.exs
config :opentelemetry_exporter,
  otlp_protocol: :http_protobuf,
  otlp_endpoint: "https://your-langfuse-host/api/public/otel",
  otlp_headers: [
    {"Authorization", "Basic " <> Base.encode64("pk-lf-...:sk-lf-...")}
  ]

Trace-level attributes via custom_context

Set well-known keys on your chain's custom_context to propagate Langfuse trace attributes onto the root chain span:

chain =
  %{llm: llm, messages: messages}
  |> LLMChain.new!()
  |> Map.put(:custom_context, %{
    langfuse_user_id: current_user.id,
    langfuse_session_id: session_id,
    langfuse_tags: ["production", "v2"],
    langfuse_metadata: %{env: "prod", feature: "chat"}
  })

These stay on the root chain span and stay strings, because that's what Langfuse reads. For attributes you want on every span, with types preserved, use :otel_attributes instead (see Attaching your own context to spans above). The two can be combined freely.

Robustness

The span handler traps and logs any exception it raises rather than letting it propagate. This matters because :telemetry permanently detaches a handler that raises — a single bad payload would otherwise silently disable tracing VM-wide for the rest of the run. Instead, that one event is skipped and tracing stays attached.

See also

• LangChain.Telemetry — the full event catalog and metadata shapes
• LangChain.OpenTelemetry — setup, teardown, and without_tracing/1
• LangChain.OpenTelemetry.Config — every configuration option
• LangChain.OpenTelemetry.MetricsHandler — emitted metric event shapes
• LangChain.OpenTelemetry.ProviderMapping — provider name normalization