off_broadway_beanstalkd v0.1.0 OffBroadwayBeanstalkd.Producer
A GenStage producer that continuously polls messages from a beanstalkd queue and acknowledge them after being successfully processed.
By default this producer uses OffBroadwayBeanstalkd.BeanstixClient
to talk to beanstalkd but
you can provide your client by implementing the OffBroadwayBeanstalkd.BeanstalkdClient
behaviour.
Options for OffBroadwayBeanstalkd.BeanstixClient
:host
- Optional. The host beanstalkd is running on, default is '127.0.0.1':port
- Optional. The port beanstalkd is running on, default is 11300:tube
- Optional. The name of the tube, default is "default":requeue
- Optional. Defines a strategy for requeuing failed messages. Possible values are::always
,:never
,:once
or can be an integer for the number of to requeue eg. if requeue is 5 a message will be tried 6 times before being deleted Default is:always
.:requeue_delay_min
- The minimum requeue delay in seconds (default:10
):requeue_delay_max
- The maximum requeue delay in seconds (default:60
)
Producer Options
These options applies to all producers, regardless of client implementation:
:receive_interval
- Optional. The duration (in milliseconds) for which the producer waits before making a request for more messages. Default is 1000.:beanstalkd_client
- Optional. A module that implements theOffBroadwayBeanstalkd.BeanstalkdClient
behaviour. This module is responsible for fetching and acknowledging the messages. Pay attention that all options passed to the producer will be forwarded to the client. It's up to the client to normalize the options it needs. Default isOffBroadwayBeanstalkd.BeanstixClient
.
Acknowledgments
In case of successful processing, the message is deleted from the queue. In case of failures, the message is released back to the ready queue or deleted depanding on the requeue option.
Batching
Bathcing with Broadway is done using the handle_batch/3
callback
Example
Broadway.start_link(MyBroadway,
name: MyBroadway,
producer: [
module:
{OffBroadwayBeanstalkd.Producer,
host: "192.168.0.10",
port: 11300,
tube: "my_queue",
requeue: :once
concurrency: 5
],
processors: [
default: []
]
)
The above configuration will set up a producer that continuously receives
messages from "my_queue"
and sends them downstream.
Retrieving Metadata
By default the following information is added to the metadata
field in the
%Message{}
struct:
id
- The job id received when the message was sent to the queue
You can access any of that information directly while processing the message:
def handle_message(_, message, _) do
job_id message.metadata.id,
# Do something with the job_id
end