Scriba.Failure (Scriba v0.1.0)

Copy Markdown View Source

Classifies a failed commit into one of three kinds, so the engine can pick a response that terminates.

This exists because "did some events succeed?" is not a usable proxy for "is this failure deterministic?". Partial success measures uniformity, not determinism, and it is wrong in both directions:

  • Pool exhaustion, a statement timeout under load, a serialization failure or a failover mid-batch all produce partial failure as pressure eases. Reading that as "deterministic" would dead-letter perfectly good events and advance past them — data loss caused by a transient blip.

  • Handler code deployed ahead of its migration produces undefined column on every event in the batch, forever. Reading uniform failure as "environmental" replays it in an infinite loop.

Postgres already publishes the answer as SQLSTATE, so the engine reads that rather than guessing. See https://www.postgresql.org/docs/current/errcodes-appendix.html.

The three kinds

  • :integrity — the event's data is bad against the current schema (SQLSTATE class 23, plus class 22 data exceptions such as numeric overflow, plus Ecto.ConstraintError and invalid changesets). Deterministic and specific to one event: it will fail identically on every redelivery. Dead-letter it and advance.

  • :transient — the database is unreachable, busy, restarting, or asked us to back off. Classes 08 (connection exceptions), 53 (insufficient resources) and 57 (operator intervention: shutdown, restart, failover, still-starting-up), plus 40001 and 40P01 (serialization failure, deadlock), 25006 (a write that landed on a read-only replica mid-failover), any Postgrex.Error carrying no SQLSTATE, and DBConnection errors. Nothing about the event is wrong. Fail and replay.

    Class 57 is here because of E3: docker stop emits 57P01 (admin_shutdown), which an earlier version of this module — listing 57014 alone out of that class — sent to :structural and halted the projection permanently over a routine database restart. 57P04 (database_dropped) is carved back out as :structural, because a dropped database is not a restarted one.

  • :structural — the schema or permissions do not match the code (class 42), and everything unrecognised. Neither response is safe here: dead-lettering destroys a projection's worth of events over a fixable deploy-ordering mistake, and replaying loops forever. Halt, loudly. A stall that announces itself is a legitimate outcome; the failure mode this library had was that it was silent.

Unrecognised failures classify as :structural deliberately. Halting on something we cannot name is recoverable by a human; guessing is not.

Your pool settings decide whether this module runs at all

DBConnection blocks on connection checkout rather than erroring, for up to :queue_target / :queue_interval. A database blip shorter than that window is absorbed beneath Scriba entirely: no error is raised, nothing is classified, and the projection simply pauses and continues.

That is good behaviour and Scriba does not try to pre-empt it. But it means pool configuration, not this module, determines where the boundary sits between "absorbed silently" and "classified and replayed" — and therefore whether the escalating backoff in Scriba.Circuit ever engages. A repo tuned with an aggressive :queue_target will reach this code on outages that a default-tuned repo never notices.

Summary

Functions

Classifies a commit failure reason.

A short label for telemetry and dead-letter rows. Returns the SQLSTATE when one is available, since that is what an operator searches for.

Types

kind()

@type kind() :: :integrity | :transient | :structural

Functions

classify(arg1)

@spec classify(term()) :: kind()

Classifies a commit failure reason.

Accepts what Scriba.Target.apply_batch/6 can report: a Postgrex.Error, an Ecto.ConstraintError, an invalid Ecto.Changeset, a DBConnection error, or any other term.

label(other)

@spec label(term()) :: String.t()

A short label for telemetry and dead-letter rows. Returns the SQLSTATE when one is available, since that is what an operator searches for.