View Source RabbitMQStream.SuperProducer behaviour (rabbitmq_stream v0.4.0-rc.3)
A Superproducer spawns a Producer process for each partition of the stream,
and uses the routing_key/2
callback to forward a publish command to the
producer of the partition.
It accepts the same options as a Producer, plus the following:
:super_stream
- the name of the super stream:partitions
- the number of partitions
All the producers use the same provided connection, and are supervised by a DynamicSupervisor.
You can optionally implement a routing_key/2
callback to compute the target
partition for a given message. By default, the partition is computed using
:erlang.phash2/2
.
Setup
To start a Superproducer, you need to make sure that each stream/partition is created beforehand. As of RabbitMQ 3.11.x and 3.12.x, this can only be done using an AMQP Client, RabbitMQ Management or with the RabbitMQ CLI..
The easiest way to do this is to use the RabbitMQ CLI:
$ rabbitmq-streams add_super_stream invoices --partitions 3
As of RabbitMQ 3.13.x, you can also create a super stream using the
RabbitMQStream.Connection.create_super_stream/4
.
Summary
Callbacks
Callback responsible for generating the routing key for a given message and partitions size, which is then used to forward the publish request to the RabbitMQStream.Producer process responsible for the partition.
Types
@type super_producer_option() :: {:super_stream, String.t()} | {:partitions, non_neg_integer()} | RabbitMQStream.Producer.option()
@type t() :: %RabbitMQStream.SuperProducer{ connection: GenServer.server(), dynamic_supervisor: module(), partitions: non_neg_integer(), producer_module: module(), producer_opts: [RabbitMQStream.Producer.option()] | nil, registry: module(), routes: term(), super_stream: String.t() }
Callbacks
@callback routing_key(message :: binary(), partitions :: non_neg_integer()) :: non_neg_integer() | binary()
Callback responsible for generating the routing key for a given message and partitions size, which is then used to forward the publish request to the RabbitMQStream.Producer process responsible for the partition.