View Source OffBroadwayMemory.Producer (off_broadway_memory v1.0.0)

A Broadway producer for OffBroadwayMemory.Buffer.

Options

  • :buffer_pid (pid/0) - Required. The buffer responsible for storing the queued messages.

  • :resolve_pending_timeout (non_neg_integer/0) - The duration (in milliseconds) of the timeout period observed between attempts to resolve any pending demand. The default value is 100.

  • :on_failure (atom/0) - The action to perform on failed messages. Options are :requeue and :discard. This can also be configured on a per-message basis with Broadway.Message.configure_ack/2. The default value is :requeue.

Example

# Start a buffer:
{:ok, buffer_pid} = OffBroadwayMemory.Buffer.start_link()

# Connect it to Broadway:
Broadway.start_link(MyBroadway,
  name: MyBroadway,
  producer: [
    module: {OffBroadwayMemory.Producer, buffer_pid: buffer_pid},
    concurrency: 1
  ],
  processors: [default: [concurrency: 50]],
)

# Push data to be processed:
OffBroadwayMemory.Buffer.push(buffer_pid, ["example", "data", "set"])

Telemetry

This library exposes the following Telemetry events:

  • [:off_broadway_memory, :receive_messages, :start] - Emitted before receiving messages from the buffer

    • Measurement:
      %{
        # The current system time in native units from
        # calling: erlang:system_time()
        system_time => integer(),
        monotonic_time => integer(),
      }
    • Metadata:
      %{
        name: atom(),
        demand: integer()
      }
  • [:off_broadway_memory, :receive_messages, :stop] - Emitted after messages have been received from the buffer and wrapped

    • Mesurement:
      %{
        # The current monotonic time minus the start monotonic time in native units
        # by calling: erlang:monotonic_time() - start_monotonic_time
        duration => integer(),
        monotonic_time => integer()
      }
    • Metadata:
      %{
        name: atom(),
        messages: [Broadway.Message.t()],
        demand: integer()
      }