LangExtract.Runner.Request (LangExtract v0.7.0)

Copy Markdown View Source

A single chunk request through the runner's budget, with the runner's retry policy.

Every attempt acquires from the Limiter first and releases after. The policy per outcome:

  • 429 — pause the limiter globally until the server's retry-after deadline and retry. Rate-limit waits never consume the chunk's retry budget: the server asked us to wait, not to give up. When retry-after is absent the pause escalates exponentially from one backoff period (capped at 30s), and after :rate_limit_retries 429s on one chunk (default 10) the chunk fails with the rate-limit error — bounded, unlike a budget, only by persistence of the 429s.
  • 5xx / transport error — jittered exponential backoff, consumes one unit of chunk_retries; budget exhausted returns the last error.
  • any other error (4xx, parse-level) — returned immediately; a bad request never gets better by retrying.

Emits [:lang_extract, :chunk, :retry] before each retry with the attempt number and a compact reason (:rate_limited | :server_error | :transport_error).

Summary

Functions

infer(limiter, client, prompt, opts)

@spec infer(GenServer.server(), LangExtract.Client.t(), String.t(), keyword()) ::
  {:ok, String.t()} | {:error, LangExtract.Provider.error()}