View Source RabbitMQStream.SuperProducer behaviour (rabbitmq_stream v0.4.0-rc.1)

A Superproducer spawns a Producer process for each partition of the stream, and uses the routing_key/2 callback to forward a publish command to the producer of the partition.

It accepts the same options as a Producer, plus the following:

  • :super_stream - the name of the super stream
  • :partitions - the number of partitions

All the producers use the same provided connection, and are supervised by a DynamicSupervisor.

You can optionally implement a routing_key/2 callback to compute the target partition for a given message. By default, the partition is computed using :erlang.phash2/2.

Setup

To start a Superproducer, you need to make sure that each stream/partition is created beforehand. As of RabbitMQ 3.11.x and 3.12.x, this can only be done using an AMQP Client, RabbitMQ Management or with the RabbitMQ CLI..

The easiest way to do this is to use the RabbitMQ CLI:

$ rabbitmq-streams add_super_stream invoices --partitions 3

As of RabbitMQ 3.13.x, you can also create a super stream using the RabbitMQStream.Connection.create_super_stream/4.

Summary

Callbacks

Callback responsible for generating the routing key for a given message and partitions size, which is then used to forward the publish request to the RabbitMQStream.Producer process responsible for the partition.

Types

Link to this type

super_producer_option()

View Source
@type super_producer_option() ::
  {:super_stream, String.t()}
  | {:partitions, non_neg_integer()}
  | RabbitMQStream.Producer.option()
@type t() :: %RabbitMQStream.SuperProducer{
  connection: GenServer.server(),
  dynamic_supervisor: module(),
  partitions: non_neg_integer(),
  producer_module: module(),
  producer_opts: [RabbitMQStream.Producer.option()] | nil,
  registry: module(),
  routes: term(),
  super_stream: String.t()
}

Callbacks

Link to this callback

routing_key(message, partitions)

View Source
@callback routing_key(message :: binary(), partitions :: non_neg_integer()) ::
  non_neg_integer() | binary()

Callback responsible for generating the routing key for a given message and partitions size, which is then used to forward the publish request to the RabbitMQStream.Producer process responsible for the partition.