Honeydew 💪🏻🍈

Build Status Hex pm

Honeydew (“Honey, do!”) is a pluggable job queue and worker pool for Elixir, focused on at-least-once execution.

defmodule MyWorker do
  def do_a_thing do
    IO.puts "doing a thing!"
  end
end

:ok = Honeydew.start_queue(:my_queue)
:ok = Honeydew.start_workers(:my_queue, MyWorker)

:do_a_thing |> Honeydew.async(:my_queue)

# => "doing a thing!"

Isolation

  • Jobs are run in isolated one-time-use processes.
  • Optionally stores immutable state loaned to each worker (a database connection, for example).
  • Initialized Worker

Strong Job Custody

  • Jobs don’t leave the queue until either they succeed, are explicitly abandoned or are moved to another queue.
  • Workers are issued only one job at a time, no batching.
  • If a worker crashes while processing a job, the job is reset and a “failure mode” (e.g. abandon, move, retry) is executed.
  • Job Lifecycle

Clusterable Components

  • Queues, workers and your enqueuing processes can exist anywhere in the BEAM cluster.
  • Global Queues

Plugability

Batteries Included

  • In-Memory Queue, for fast processing of recreatable jobs.
  • Mnesia Queue, for persistence and simple distribution scenarios.
  • Ecto Queue, to turn an Ecto schema into its own work queue, using your database.
  • Can optionally heal the cluster after a disconnect or downed node when using a Global Queue.

Easy API

  • Jobs are enqueued using async/3 and you can receive replies with yield/2, somewhat like Task.
  • API

Ecto Queue

The Ecto Queue is designed to painlessly turn your Ecto schema into a queue, using your repo as the backing store.

  • You don’t need to explicitly enqueue jobs, that’s handled for you (for example, sending a welcome email when a new User is inserted).
  • Eliminates the possibility of your database and work queue becoming out of sync
  • As the database is the queue, you don’t need to run a separate queue node.
  • You get all of the high-availability, consistency and distribution semantics of your chosen database.

Check out the included example project, and its README.

Getting Started

In your mix.exs file:

defp deps do
  [{:honeydew, "~> 1.2.8"}]
end

tl;dr

README

The rest of the README is broken out into slightly more digestible sections.

Also, check out the README files included with each of the examples.