LokiLoggerHandler (LokiLoggerHandler v0.4.0)

Elixir Logger handler for Grafana Loki.

This library implements an Erlang :logger handler that buffers logs and sends them to Loki in batches. It supports:

Configurable label extraction for Loki stream labels
Structured metadata (Loki 2.9+)
Two storage strategies: disk (CubDB) or memory (ETS)
Dual threshold batching (time and size)
Exponential backoff on failures
Multiple handler instances for different Loki endpoints

Quick Start

# Attach a handler
LokiLoggerHandler.attach(:my_handler,
  loki_url: "http://localhost:3100",
  labels: %{
    app: {:static, "myapp"},
    env: {:metadata, :env},
    level: :level
  },
  structured_metadata: [:request_id, :user_id]
)

# Now use Logger as usual
require Logger
Logger.info("Hello Loki!", request_id: "abc123")

# Later, detach if needed
LokiLoggerHandler.detach(:my_handler)

Configuration Options

Option	Type	Default	Description
`:loki_url`	string	required	Loki push API base URL
`:storage`	atom	`:disk`	Storage strategy: `:disk` (CubDB) or `:memory` (ETS)
`:labels`	map	`%{level: :level}`	Label extraction config
`:structured_metadata`	list	`[]`	Metadata keys for Loki structured metadata
`:data_dir`	string	`"priv/loki_buffer/<id>"`	CubDB storage directory (disk only)
`:batch_size`	integer	100	Max entries per batch
`:batch_interval_ms`	integer	5000	Max time between batches
`:max_buffer_size`	integer	10000	Max buffered entries before dropping
`:backoff_base_ms`	integer	1000	Base backoff on failure
`:backoff_max_ms`	integer	60000	Max backoff time
`:connect_options`	keyword	`[]`	Connection options passed to Req.post

Label Configuration

Labels are configured as a map where keys are the Loki label names and values specify how to extract the label value:

:level - Use the log level
{:metadata, key} - Extract from log metadata
{:static, value} - Use a static value

Example:

labels: %{
  app: {:static, "myapp"},
  environment: {:metadata, :env},
  level: :level,
  node: {:metadata, :node}
}

Structured Metadata (Loki 2.9+)

Structured metadata allows attaching key-value pairs that aren't indexed as labels but can still be queried. Specify a list of metadata keys to extract:

structured_metadata: [:request_id, :user_id, :trace_id, :span_id]

Telemetry

The library emits telemetry events for monitoring buffer state:

[:loki_logger_handler, :buffer, :insert] - After a log entry is buffered
[:loki_logger_handler, :buffer, :remove] - After entries are sent and removed

Both events include:

Measurements: %{count: integer} - Buffer size after the operation
Metadata: %{handler_id: atom, storage: :cub | :ets}

A further event is emitted when an event cannot be formatted:

[:loki_logger_handler, :format, :error] - When formatting raises. A best-effort fallback entry is buffered instead, so the handler is never removed from :logger by a single bad event.
- Measurements: %{count: 1}
- Metadata: %{handler_id: atom, reason: term, stacktrace: Exception.stacktrace()}

Example:

:telemetry.attach(
  "loki-buffer-monitor",
  [:loki_logger_handler, :buffer, :insert],
  fn _event, %{count: count}, %{handler_id: id}, _config ->
    IO.puts("Handler #{id} buffer size: #{count}")
  end,
  nil
)

Summary

Types

handler_id()

option()

Functions

attach(handler_id, opts)

Attaches a new Loki logger handler.

detach(handler_id)

Detaches a Loki logger handler.

flush(handler_id)

Forces an immediate flush of all pending logs for a handler.

get_config(handler_id)

Returns the current configuration for a handler.

list_handlers()

Lists all attached Loki logger handlers.

update_config(handler_id, opts)

Updates the configuration of an existing handler.

Types

handler_id()

@type handler_id() :: atom()

option()

@type option() ::
  {:loki_url, String.t()}
  | {:storage, :disk | :memory}
  | {:labels, map()}
  | {:structured_metadata, [atom()]}
  | {:data_dir, String.t()}
  | {:batch_size, pos_integer()}
  | {:batch_interval_ms, pos_integer()}
  | {:max_buffer_size, pos_integer()}
  | {:backoff_base_ms, pos_integer()}
  | {:backoff_max_ms, pos_integer()}
  | {:connect_options, keyword()}

Functions

attach(handler_id, opts)

@spec attach(handler_id(), [option()]) :: :ok | {:error, term()}

Attaches a new Loki logger handler.

Parameters

handler_id - A unique atom identifier for this handler
opts - Configuration options (see module docs)

Returns

:ok on success
{:error, reason} on failure

Examples

LokiLoggerHandler.attach(:default,
  loki_url: "http://localhost:3100",
  labels: %{app: {:static, "myapp"}, level: :level}
)

detach(handler_id)

@spec detach(handler_id()) :: :ok | {:error, term()}

Detaches a Loki logger handler.

Parameters

handler_id - The handler identifier used when attaching

Returns

:ok on success
{:error, reason} if the handler doesn't exist

Examples

LokiLoggerHandler.detach(:default)

flush(handler_id)

@spec flush(handler_id()) :: :ok | {:error, term()}

Forces an immediate flush of all pending logs for a handler.

Useful before application shutdown to ensure all logs are sent.

Parameters

handler_id - The handler identifier

Returns

:ok on success
{:error, reason} on failure

get_config(handler_id)

@spec get_config(handler_id()) :: {:ok, map()} | {:error, term()}

Returns the current configuration for a handler.

Parameters

handler_id - The handler identifier

Returns

{:ok, config} with the handler configuration
{:error, reason} if the handler doesn't exist

list_handlers()

@spec list_handlers() :: [handler_id()]

Lists all attached Loki logger handlers.

Returns

A list of handler IDs that are using this handler module.

update_config(handler_id, opts)

@spec update_config(handler_id(), [option()]) :: :ok | {:error, term()}

Updates the configuration of an existing handler.

Parameters

handler_id - The handler identifier
opts - New configuration options to merge

Returns

:ok on success
{:error, reason} on failure