Derives a stable key from a job's payload.
Used for unique keys, and for batch keys once those exist. The whole
payload hashes by default; :only and :except narrow it to the
parts that should decide identity:
# Two enqueues with the same user and template are the same job.
Zizq.PayloadHasher.new!(only: [".user_id", ".template"])
# Everything except a timestamp that changes every call.
Zizq.PayloadHasher.new!(except: [".requested_at"])Paths are jq-flavoured. Declared on a job module they are parsed when that module compiles, so a malformed path is a build failure and no enqueue pays to parse it:
use Zizq.JobKind,
type: "send_email",
unique_key: {:payload, only: [".user_id", ".template"]},
unique_while: :queuedPaths
| Path | Selects |
|---|---|
"." | the whole payload |
".user_id" | a key |
".user.id" | a nested key |
".items[0]" | an array element |
".[0]" | an element of a root array |
~s(.["dotted.key"]) | a key containing a dot |
A path that matches nothing is skipped rather than treated as
null, so payloads that omit an optional field still hash to the
same key as those that never had it.
How the digest is built
The payload is first round-tripped through JSON, collapsing structs,
atom keys, DateTimes and charlists into what the server actually
stores. The result is then streamed into SHA-256 as canonical JSON:
object keys sorted, arrays in order, with {, }, [, ], :
and , markers so that [1, 2] and [12] cannot collide.
:erlang.term_to_binary/2 with :deterministic would be one line
and much faster, but it hashes the Erlang representation rather than
the value: %{"a" => 1} and %{a: 1} differ, as do 1 and 1.0,
none of which survives the trip to the server. It is also not frozen
across OTP releases.
Stability
The digest is stable within Elixir across releases of this
client, which is what uniqueness depends on. It is not guaranteed to
equal the digest another language's client computes for the same
payload: JSON does not distinguish 1 from 1.0 but Elixir does, so
a float that another runtime would render as 1 renders here as
1.0. Dedupe within one producer, not across producers in different
languages.
Summary
Functions
Build the bare hex digest for payload, without a type prefix.
Build the key for a job of type carrying payload.
Build a hasher, parsing its paths.
Parse a jq-flavoured path into steps.
Types
@type step() :: {:key, String.t()} | {:index, non_neg_integer()}
Functions
Build the bare hex digest for payload, without a type prefix.
Build the key for a job of type carrying payload.
Build a hasher, parsing its paths.
Options
:only— a path or list of paths whose values decide the key.:except— a path or list of paths to leave out. Cannot be combined with:only.:prefix— prefix the key with the job type and a:. Defaults totrue, which keeps two kinds of job with identical payloads from colliding.
Parse a jq-flavoured path into steps.
Called while a job module compiles, so a bad path fails the build.