Prometheus Metrics

View Source

See also: Observability Guide

For a complete guide covering tracing, metrics, and logging with exporters, see the Observability Guide.

Barrel DocDB records metrics in-process using the instrument library and can render them in Prometheus text format.

Reading Metrics

barrel_docdb has no built-in HTTP server. Render the current metrics in Prometheus text format in-process:

Text = barrel_metrics:export_text().
%% Prometheus exposition format, ready to serve or log

To expose metrics over HTTP for a Prometheus scrape, run the barrel_server app, which serves this output on its metrics endpoint. See the umbrella REST server guide (docs/guides/rest-server.md). Metrics can also be pushed to an OTLP collector; see Observability.

Available Metrics

Document Operations

MetricTypeLabelsDescription
barrel_doc_operationsCounterdb, operationTotal document operations
barrel_doc_operation_duration_secondsHistogramdb, operationOperation latency

Operations: put, get, delete

Example:

barrel_doc_operations{db="mydb",operation="put"} 1234
barrel_doc_operations{db="mydb",operation="get"} 5678
barrel_doc_operation_duration_seconds_bucket{db="mydb",operation="get",le="0.001"} 5000

Query Operations

MetricTypeLabelsDescription
barrel_query_operationsCounterdbTotal query operations
barrel_query_duration_secondsHistogramdbQuery latency
barrel_query_results_countHistogramdbResults per query

Example:

barrel_query_operations{db="mydb"} 500
barrel_query_duration_seconds_bucket{db="mydb",le="0.01"} 450

Replication

MetricTypeLabelsDescription
barrel_replication_docsCounterdirectionDocuments replicated
barrel_replication_errorsCountertask_idReplication errors
barrel_replication_lag_secondsGaugetask_idReplication lag
barrel_replication_activeGaugetask_idActive status (1/0)

Example:

barrel_replication_docs{direction="push"} 10000
barrel_replication_errors{task_id="rep-123"} 2

Storage

MetricTypeLabelsDescription
barrel_db_documents_totalGaugedbDocument count
barrel_db_size_bytesGaugedbDatabase size
barrel_db_attachments_totalGaugedbAttachment count

HTTP request metrics

Metrics for HTTP requests (method, path, status, latency) are produced by barrel_server, not barrel_docdb, since barrel_docdb has no HTTP surface of its own.

Grafana Dashboard

Example queries for common dashboards:

Document Operation Rate

rate(barrel_doc_operations[5m])

P99 Document Latency

histogram_quantile(0.99, rate(barrel_doc_operation_duration_seconds_bucket[5m]))

Documents per Second

rate(barrel_doc_operations{operation="put"}[5m])

Query Throughput

rate(barrel_query_operations[5m])

Replication Throughput

increase(barrel_replication_docs[1m])

Alerting Examples

High Latency

- alert: BarrelHighLatency
  expr: |
    histogram_quantile(0.99, rate(barrel_doc_operation_duration_seconds_bucket[5m])) > 0.1
  for: 5m
  labels:
    severity: warning
  annotations:
    summary: "High P99 latency on Barrel DocDB"

Replication Errors

- alert: BarrelReplicationErrors
  expr: increase(barrel_replication_errors[5m]) > 0
  for: 1m
  labels:
    severity: warning
  annotations:
    summary: "Replication errors detected"

Configuration

Metrics are always recorded internally. Set the exporter to none (the default) to suppress external export, or to console / otlp to push them out. See Observability for details.

{barrel_docdb, [
    {metrics, [
        {exporter, none}  %% or console | otlp
    ]}
]}.