Kiq v0.1.0 Kiq behaviour View Source

Kiq is a robust and extensible job processing queue that aims for compatibility with Sidekiq Enterprise.

Job queuing, processing and reporting are all built on GenStage. That means maximum parallelism with the safety of backpressure as jobs are processed.

Usage

Kiq isn’t an application that must be started. Similarly to Ecto, you define one or more Kiq modules within your application. This allows multiple supervision trees with entirely different configurations.

Define a Kiq module for your application:

defmodule MyApp.Kiq do
  use Kiq, queues: [default: 25, events: 50]

  @impl Kiq
  def init(_reason, opts) do
    for_env = Application.get_env(:my_app, :kiq)

    opts =
      opts
      |> Keyword.merge(for_env)
      |> Keyword.put(:client_opts, [redis_url: System.get_env("REDIS_URL")])

    {:ok, opts}
  end
end

Include the module in your application’s supervision tree:

defmodule MyApp.Application do
  @moduledoc false

  use Application

  alias MyApp.{Endpoint, Kiq, Repo}

  def start(_type, _args) do
    children = [
      {Repo, []},
      {Endpoint, []},
      {Kiq, []}
    ]

    Supervisor.start_link(children, strategy: :one_for_one, name: MyApp.Supervisor)
  end
end

TODO: Configuration TODO: Workers TODO: Testing TODO: Error Handling

Link to this section Summary

Callbacks

Clear all enqueued, scheduled and backup jobs

Enqueue a job to be processed asynchronously

A callback executed when the supervision tree is started and possibly when configuration is read

Starts the client and possibly the supervision tree, returning {:ok, pid} when startup is successful

Link to this section Types

Link to this type job_args() View Source
job_args() :: map() | Keyword.t() | Kiq.Job.t()
Link to this type job_opts() View Source
job_opts() :: [in: pos_integer(), at: DateTime.t()]

Link to this section Callbacks

Link to this callback clear_all() View Source
clear_all() :: :ok

Clear all enqueued, scheduled and backup jobs.

All known queues are cleared, even if they aren’t listed in the current configuration.

Link to this callback enqueue(job_args, job_opts) View Source
enqueue(job_args(), job_opts()) :: {:ok, Kiq.Job.t()} | {:error, Exception.t()}

Enqueue a job to be processed asynchronously.

Jobs can be enqueued from Job structs, maps or keyword lists.

Options

  • in - The amount of time in seconds to wait before processing the job. This must be a positive integer.
  • at - A specific DateTime in the future when the job should be processed.

Examples

# Enqueue a job to be processed immediately
MyJob.new([1, 2]) |> MyKiq.enqueue()

# Enqueue a job in one minute
MyJob.new([1, 2]) |> MyKiq.enqueue(in: 60)

# Enqueue a job some time in the future
MyJob.new([1, 2]) |> MyKiq.enqueue(at: ~D[2020-09-20 12:00:00])

# Enqueue a job from scratch, without using a worker module
MyKiq.enqueue(class: "ExternalWorker", args: [1])
Link to this callback init(reason, opts) View Source
init(reason :: :supervisor, opts :: Keyword.t()) :: {:ok, Keyword.t()} | :ignore

A callback executed when the supervision tree is started and possibly when configuration is read.

The first argument is the context of the callback being invoked. In most circumstances this will be :supervisor. The second argument is a keyword list of the combined options passed to use/1 and start_link/1.

Application configuration is not passed into the init/2 callback. To use application config the callback must be overridden and merged manually.

Link to this callback start_link(opts) View Source
start_link(opts :: Keyword.t()) :: Supervisor.on_start()

Starts the client and possibly the supervision tree, returning {:ok, pid} when startup is successful.

Returns {:error, {:already_started, pid}} if the tree is already started or {:error, term} in case anything else goes wrong.

Options

Any options passed to start_link will be merged with those provided in the use block.