Oban plugin that keeps workflows healthy.
Add it to your Oban plugins: list like any other plugin:
config :my_app, Oban,
plugins: [
Oban.Plugins.Pruner,
{Baton.Plugin, interval: :timer.seconds(60)}
]The plugin runs two independent timers so latency-sensitive health work and bulk retention work can move at different cadences.
Health sweep — every :interval (default 60s)
Rescues orphaned waiting jobs. A job stuck in
available/scheduled/retryablewhose dependency's Oban job has been pruned before completing will never resolve on its own. The plugin finds these (viaworkflow_nodesjoined tooban_jobs) and cancels them withOban.cancel_all_jobs/2— a public API, not a manual state write.Emits failure telemetry. For any workflow that has fully terminated with at least one cancelled/discarded step, it emits
[:baton, :workflow, :failed]so you can alert.Backstops the terminal notification. For those same failed workflows it calls
Baton.Completion.announce/2, which broadcasts the one-shot{:workflow_finished, _}event (and[:baton, :workflow, :finished]telemetry) for workflows that settled without a clean worker return — a hard crash or an Oban kill, whereBaton.Workernever got to announce. An atomic claim makes this a no-op for workflows already announced by the worker, so no duplicate event is sent.
The crash case (a step that raised or exhausted its retries) is now caught
promptly by Baton.CompletionReporter, an Oban-telemetry handler attached
app-wide by Baton.Application. The sweep's backstop therefore only matters
for workflows killed entirely outside job execution (e.g. an external
Oban.cancel_all_jobs), which lets :interval be lengthened without delaying
the common case.
Prune sweep — every :prune_interval (opt-in, default 5 min)
- Prunes Baton's own tables. Baton's rows
(
workflow_nodes,workflow_step_stats,workflow_debug_logs,workflow_completions) have no foreign key tooban_jobs, so they don't vanish when Oban'sPrunerdeletes the jobs. Withprune: true, the prune sweep deletes Baton rows once their backing Oban job is gone (seeBaton.Retention). This is off by default and runs on its own, slower timer so the cheap health sweep isn't coupled to bulk deletes.
Options
:interval— health sweep interval in milliseconds (default: 60_000):prune— delete orphaned Baton rows (default:false):prune_interval— prune sweep interval in milliseconds (default: 300_000); only used whenprune: true:prune_limit— max rows deleted per table per prune sweep (default: 10_000):debug_log_max_age— optional; also deleteworkflow_debug_logsolder than this many seconds, regardless of job state (they are the largest rows and usually want a shorter retention than the rest)
Example
{Baton.Plugin,
interval: :timer.seconds(60),
prune: true,
prune_interval: :timer.minutes(5),
debug_log_max_age: :timer.hours(24) |> div(1000)}Telemetry
[:baton, :plugin, :rescued]— measurements%{count}, metadata%{job_ids}[:baton, :plugin, :pruned]— measurements%{count}, metadata is the per-table counts, emitted only whenprune: trueremoved at least one row[:baton, :workflow, :failed]— measurements%{failed_count, total_count}, metadata%{workflow_id, workflow_label}
Summary
Functions
Returns a specification to start this module under a supervisor.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.