Errol
An opinionated framework to run and orchestrate RabbitMQ consumers.
Index
Installation
The package can be installed by adding errol
to your list of dependencies in mix.exs
:
def deps do
[{:errol, "~> 0.1.0"}]
end
You should also update your application list to include :errol
:
def application do
[applications: [:errol]]
end
Documentation can be found at https://hexdocs.pm/errol.
Usage
To bind consumers to queue, you can use the Errol.Wiring
module:
defmodule Sample.Wiring do
use Wiring
connection "amqp://guest:guest@localhost"
@exchange "/users"
@exchange_type :topic
# You can pass a reference to a function with arity of 1
consume("account_created", "users.account.created", &UsersConsumer.account_created/1)
# or even an anonymous function
consume("account_updated", "users.account.updated", fn message -> ... end)
end
For more complex setups, you can add middleware and group different consumers for more granularity.
defmodule Sample.Wiring do
use Wiring
connection "amqp://guest:guest@localhost"
@exchange "/users"
@exchange_type :topic
# Use pipe_before/1, pipe_after/1 or pipe_error/1 to run middleware functions
# middlewares declared outside of a group will run for every consumer
pipe_after(&Sample.StatisticsMiddleware.track/2)
# Use the `group` macro to group consumers with specific middleware
group :account do
# This middleware will run only for consumers in the :account group
pipe_before(&Errol.Middleware.Json.parse/2)
consume("account_created", "users.account.created", &UsersConsumer.account_created/1)
consume("account_updated", "users.account.updated", fn message -> ... end)
end
group :photos do
pipe_before(&Sample.ImagesMiddleware.uncompress/2)
consume("profile_photo_uploaded", "users.profile.photo.uploaded", fn message -> ... end)
end
end
At this point, the only thing left is to run Sample.Wiring
as a supervisor in your application.ex
file:
defmodule Sample.Application do
use Application
def start(_type, _args) do
import Supervisor.Spec
children = [
supervisor(Sample.Wiring, []),
...
]
opts = [strategy: :one_for_one, name: Sample.Supervisor]
Supervisor.start_link(children, opts)
end
end
Voilà! This will spin up the following supervision tree:
--------------------
| Sample.Application |
--------------------
|
|
---------------
| Sample.Wiring |
---------------
|
_____________________________|_____________________________
| | |
| | |
--------------------------- --------------------------- -------------------------
| :account_created_consumer | | :account_updated_consumer | | :profile_photo_uploaded |
--------------------------- --------------------------- -------------------------
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
New monitored process per each message received
Compatibility with other AMQP implementations exists but is not guaranteed (at least for now 😁).
Running locally
Clone the repository
git clone git@github.com:uesteibar/errol.git
Install dependencies
cd errol
mix deps.get
To run the tests (you will need docker installed)
./scripts/test_prepare.sh
mix test
Roadmap
- [ ] Allowing to specify the number of retries. For now, when a message fails to be processed it will be redelivered only once. After that the message will be just rejected.
- [ ] Adding a middleware to ensure duplicated messages are not consumed more than once.
- [ ] Allowing to specify number of workers per consumer. Poolboy would come handy here.
- [ ] Handle RabbitMQ outages, following the great explanation in the amqp hex documentation.
- [ ] Publishing messages.
Contributing
Pull requests are always welcome =)
The project uses standard-changelog to update the Changelog with each commit message and upgrade the package version.
For that reason every contribution must have a title and body that follows the conventional commits standard conventions (e.g. feat(consumer): Consume it all
).
To make this process easier, you can do the following:
Install commitizen
and cz-conventional-changelog
globally
npm i -g commitizen cz-conventional-changelog
Save cz-conventional-changelog
as default
echo '{ "path": "cz-conventional-changelog" }' > ~/.czrc
Instead of git commit
, you can now run
git cz
and follow the instructions to generate the commit message.
Credit
🎉 Special thanks to @pma for the amazing work on the amqp hex.