Base module for workflow steps that call an LLM API.

Extends Baton.Worker with LLM-appropriate defaults:

• Configurable timeout/1 — defaults to Baton's configured ten-minute cap, overridable per worker with the :timeout option (in milliseconds). LLM calls can run long, so workers that need more time should explicitly declare a generous cap — for example use Baton.LLMWorker, timeout: :timer.minutes(5).
• :llm queue — dedicated queue so LLM jobs don't compete with fast background jobs for concurrency slots
• Jitter backoff — spreads retries across a window to avoid thundering herd against the LLM API rate limiter when multiple jobs fail simultaneously
• Idempotency — inherited from Baton.Worker; a retried job that already stored a result returns it immediately without re-calling the API
• Automatic stats recording — if your result map includes an "llm_usage" key, it is stripped out and written to workflow_step_stats automatically

Usage

defmodule MyApp.Workers.Summarize do
  use Baton.LLMWorker

  @impl true
  def perform_workflow(%Oban.Job{args: args} = job) do
    start = System.monotonic_time(:millisecond)

    case MyApp.LLM.complete(build_prompt(args)) do
      {:ok, response} ->
        latency = System.monotonic_time(:millisecond) - start

        {:ok, %{
          # Your actual result — passed to downstream steps
          text: response.text,

          # Picked up automatically by LLMWorker, stripped from result,
          # written to workflow_step_stats. Never seen by downstream steps.
          llm_usage: %{
            model:               response.model,
            input_tokens:        response.usage.input_tokens,
            output_tokens:       response.usage.output_tokens,
            cache_read_tokens:   response.usage.cache_read_input_tokens,
            cache_write_tokens:  response.usage.cache_creation_input_tokens,
            latency_ms:          latency
          }
        }}

      {:error, %{status: 429}} ->
        {:snooze, 30}

      {:error, reason} ->
        {:error, reason}
    end
  end
end

Overriding defaults

All Oban.Worker options can still be overridden at the use site:

use Baton.LLMWorker,
  max_attempts: 5,
  timeout: :timer.minutes(10),
  priority: 1

Queue config

config :baton, Oban,
  queues: [default: 20, llm: 5]

Backoff behaviour

Exponential backoff with ±50% uniform jitter, driven by how many times the job has genuinely failed:

• 1st failure → retry in ~8–24s
• 2nd failure → retry in ~12–34s
• 3rd failure → retry in ~21–63s
• 4th failure → retry in ~40–118s

Failures, not attempts. Baton waits on dependencies by snoozing, and Oban counts a snooze as an attempt — a step deep in a sequential fan-out can reach attempt 70 before it first executes. Reading attempt here would price that step's first failure as its 71st and defer the retry for days. See Baton.Backoff.

A flow node overrides this curve entirely by declaring its own retry_backoff_seconds (Baton.Flow.NodeSpec) — backoff/1 checks Baton.Backoff.node_backoff/1 first and only falls back to the jittered curve above when the node left it unset. This is the exponential curve applied to every LLM flow node before that existed, and still the default for a job whose failure might mean something is actually wrong (a truncation, a malformed response). A node whose guards resample often against cheap, expected misses is the case for opting out of it.