Baton.Backoff (Baton v0.27.4)

Copy Markdown View Source

A snooze-free view of a job's retry counters.

Baton waits on dependencies by snoozing, and Oban counts a snooze as an attempt: snooze_job/3 increments attempt and max_attempts, so the remaining retry budget survives untouched. That is the right bookkeeping for a budget, but it means attempt no longer answers the question a backoff curve is actually asking — "how many times has this run and failed?"

The gap is not small. A step deep in a sequential fan-out snoozes once per poll until every predecessor completes, so it can reach attempt 70 before it first executes. Read literally, its first genuine failure looks like the 71st, and an exponential curve turns a retry that should land in seconds into a wait measured in days. The run is not dead, but nothing distinguishes it from one that is.

deflate/1 rebuilds the counters the job would have carried had it never snoozed, so a Oban.Worker.backoff/1 callback can go on reading attempt and max_attempts as it always did.

def backoff(%Oban.Job{} = job) do
  job |> Baton.Backoff.deflate() |> Oban.Worker.backoff()
end

Summary

Functions

Rewrite attempt and max_attempts as though the job had never snoozed.

How many times the job has genuinely failed, counting the failure being recorded right now.

The Oban.Worker.backoff/1 a non-LLM flow node gets: node_backoff/1 when the node declared one, otherwise deflate/1 piped through Oban's own default exponential curve — the curve every Baton.Worker used before per-node backoff existed. Baton.LLMWorker has its own fallback (jittered_backoff/1) and calls node_backoff/1 directly rather than through this function.

A flow node's own retry_backoff_seconds (Baton.Flow.NodeSpec), read straight from the node config carried in job.args["flow_node"], plus a few seconds of jitter — or nil when the node declared none, true of every job whose args carry no "flow_node" at all (every non-flow Baton worker, and any flow node that left the field unset).

Functions

deflate(job)

@spec deflate(Oban.Job.t()) :: Oban.Job.t()

Rewrite attempt and max_attempts as though the job had never snoozed.

attempt becomes the genuine failure count. max_attempts drops by the number of snoozes, which preserves the remaining budget exactly — snoozing raised both counters by one apiece, so subtracting the snoozes from one and rebasing the other leaves max_attempts - attempt unchanged. Callers that weigh attempt against budget — Oban's own default backoff clamps on the attempt / max_attempts ratio — therefore see the ratio the job would have had on a run with no dependencies to wait for.

failures(job)

@spec failures(Oban.Job.t()) :: pos_integer()

How many times the job has genuinely failed, counting the failure being recorded right now.

Oban calls Oban.Worker.backoff/1 before it appends the current failure to errors — the executor computes the backoff, then error_job/3 writes errors = errors || [current] in the same update. So the failure in flight is present as unsaved_error and absent from errors, and the count this function wants is length(errors) + 1.

for_job(job)

@spec for_job(Oban.Job.t()) :: pos_integer()

The Oban.Worker.backoff/1 a non-LLM flow node gets: node_backoff/1 when the node declared one, otherwise deflate/1 piped through Oban's own default exponential curve — the curve every Baton.Worker used before per-node backoff existed. Baton.LLMWorker has its own fallback (jittered_backoff/1) and calls node_backoff/1 directly rather than through this function.

def backoff(%Oban.Job{} = job), do: Baton.Backoff.for_job(job)

node_backoff(job)

@spec node_backoff(Oban.Job.t()) :: pos_integer() | nil

A flow node's own retry_backoff_seconds (Baton.Flow.NodeSpec), read straight from the node config carried in job.args["flow_node"], plus a few seconds of jitter — or nil when the node declared none, true of every job whose args carry no "flow_node" at all (every non-flow Baton worker, and any flow node that left the field unset).

Both flow-worker backoff callbacks check this first — Baton.Worker's (used by Baton.Flow.Workers.Action and any code-defined non-LLM worker) and Baton.LLMWorker's (used by Baton.Flow.Workers.LLM, and by every Baton.LLMStep) — before falling through to their own default curve, so a node opts into a short, roughly flat delay regardless of which worker runs it.