Batching configuration: fold an enqueue into an existing :ready job
rather than creating a new one.
Say where the batch accumulates and how large it may get, and the rest follows:
# Cap `.device_ids` at 100 entries per batch.
Zizq.BatchConfig.new!(limit: 100, path: ".device_ids")
# The whole payload is the batch, and is an array.
Zizq.BatchConfig.new!(limit: 1_000)Declared on a job module, an enqueue then carries only its own contribution and the server merges it:
defmodule MyApp.PushBatch do
use Zizq.JobKind,
type: "push",
queue: "push",
batch: [limit: 100, path: ".device_ids"]
@impl Zizq.JobKind
def perform(%{"device_ids" => ids}), do: MyApp.Push.deliver(ids)
end
MyApp.PushBatch.new(%{"device_ids" => [id], "platform" => "apple"})
|> Zizq.enqueue(MyApp.Zizq)What gets generated
| Option | Becomes |
|---|---|
:limit and :path | the :when predicate |
:path, :dedup, :sorted | the :fold expression |
:path | the default :key, hashing everything except the batch |
:key defaulting to the payload minus the batch path is what makes
the common case work without saying anything: two enqueues alike in
every respect but what they contribute belong in the same batch.
In the example above, platform decides the batch and device_ids
accumulates into it.
:dedup folds with jq's unique, :sorted with sort. unique
also sorts, so :dedup subsumes :sorted.
Options
:limit— maximum combined length at:pathbefore the batch is sealed and a new one starts.:path— jq path to the value that accumulates. Defaults to".", the whole payload.:key— override the derived key with a string, or anotherZizq.PayloadHasher.:dedup— fold withunique.:sorted— fold withsort.
Writing the expressions by hand
:key, :when and :fold can be given directly instead, for folds
the templates do not cover — accumulating into a map, say, or
counting rather than appending:
Zizq.BatchConfig.new!(
key: "digest:tenant-42",
when: "$existing.count < 100",
fold: "$existing | .count += 1 | .ids += $new.ids"
)Both are jq, evaluated with $existing bound to the batch's current
payload and $new to the incoming one. :limit and :path cannot
be mixed with them, since they would be generating the same fields.
Batching is a Pro-licensed feature; without one the server responds
403, which surfaces as %Zizq.Error{reason: :forbidden}.
The configuration is returned on job reads, and the first enqueue's
:when and :fold govern the whole batch — so reading back what is
stored is how a "first wins" surprise gets diagnosed.
Summary
Functions
Build a batch configuration from a keyword list or map.
Types
@type t() :: %Zizq.BatchConfig{ fold: String.t(), key: String.t() | Zizq.PayloadHasher.t(), when: String.t() }