Honeydew 💪🏻🍈
Honeydew (“Honey, do!”) is a pluggable job queue and worker pool for Elixir, focused on at-least-once execution.
defmodule MyWorker do
def do_a_thing do
IO.puts "doing a thing!"
end
end
:ok = Honeydew.start_queue(:my_queue)
:ok = Honeydew.start_workers(:my_queue, MyWorker)
:do_a_thing |> Honeydew.async(:my_queue)
# => "doing a thing!"
Isolation
- Jobs are run in isolated one-time-use processes.
- Optionally stores immutable state loaned to each worker (a database connection, for example).
- Initialized Worker
Strong Job Custody
- Jobs don’t leave the queue until either they succeed, are explicitly abandoned or are moved to another queue.
- Workers are issued only one job at a time, no batching.
- If a worker crashes while processing a job, the job is reset and a “failure mode” (e.g. abandon, move, retry) is executed.
- Job Lifecycle
Clusterable Components
- Queues, workers and your enqueuing processes can exist anywhere in the BEAM cluster.
- Global Queues
Plugability
- Queues, workers, dispatch strategies, failure modes and success modes are all plugable with user modules.
- No forced dependency on external queue services.
Batteries Included
- In-Memory Queue, for fast processing of recreatable jobs.
- Mnesia Queue, for persistence and simple distribution scenarios.
- Ecto Queue, to turn an Ecto schema into its own work queue, using your database.
- Can optionally heal the cluster after a disconnect or downed node when using a Global Queue.
Easy API
Ecto Queue
The Ecto Queue is designed to painlessly turn your Ecto schema into a queue, using your repo as the backing store.
- You don’t need to explicitly enqueue jobs, that’s handled for you (for example, sending a welcome email when a new User is inserted).
- Eliminates the possibility of your database and work queue becoming out of sync
- As the database is the queue, you don’t need to run a separate queue node.
- You get all of the high-availability, consistency and distribution semantics of your chosen database.
Check out the included example project, and its README.
Getting Started
In your mix.exs file:
defp deps do
[{:honeydew, "~> 1.2.8"}]
end
tl;dr
- Check out the examples.
- Enqueue jobs with
Honeydew.async/3
. - Receive responses with
Honeydew.yield/2
. - Emit job progress with
progress/1
- Queue/Worker status with
Honeydew.status/1
- Suspend and resume with
Honeydew.suspend/1
andHoneydew.resume/1
- List jobs with
Honeydew.filter/2
- Move jobs with
Honeydew.move/2
- Cancel jobs with
Honeydew.cancel/2
README
The rest of the README is broken out into slightly more digestible sections.
Also, check out the README files included with each of the examples.