Overview
Roger manages job queues in partitions. each partition contains one
or more queues. The unit of work is a job; defined by a module
which implements Roger.Job
. Jobs are enqueued into a queue, and
picked up by the partition's consumer, which spawns a supervised
process, the worker (Roger.Partition.Worker
). This worker then
calls the perform/1
function on the job's module.
Jobs
Jobs are Elixir module which implement Roger.Job
:
defmodule TestJob do
use Roger.Job
def perform(_args) do
# perform some work here...
end
end
The module has one required callback function, perform/1
, which gets
called to do the job's work. To execute the job, it needs to be
created and then enqueued in a partition:
{:ok, job} = Roger.Job.create(TestJob, %{"argument": 1})
Roger.Job.enqueue(job, "myapp")
By default, a job gets executed on the queue named default
. This
example also assumes the "myapp" partition exists. See below on how to
define a partition.
On more information on a job's properties, see the Roger.Job
module.
Partitions
To provide process isolation and fair scheduling, Roger groups its work queues by what are called partitions. Each partition can have zero or more queues associated with it, with a separate concurrency level per queue.
By default, when the :roger
OTP partition starts, it does not have
any partitions running. To always start a certain partition when
Roger starts, do the following in config.exs
:
config :roger, :partitions,
myapp: [default: 10, other: 5]
To start a partition at runtime, run:
Roger.Partition.start("myapp", [default: 10, other: 5])
This starts myapp
with two queues, one with maximum 10 concurrent
workers, and one with 5. As soon as you've done this, and the
partition is running, you can open
the RabbitMQ management console to
verify that there indeed are two queues, named myapp-default
and
myapp-other
.
In this web interface, when you click on one of the queues, you'll see that it has one consumer attached to it. You could start a second Elixir node (clustered with the first) now and see that there are now two consumers on each queue. The consumer's "prefetch count" field corresponds to the concurrency level of the queue.