LangEx.LLM.Resilient (LangEx v0.11.3)

Copy Markdown View Source

Retry-aware LLM wrapper with backoff, telemetry hooks, and fallback.

Wraps any LangEx.LLM provider with automatic retries on transient failures (rate limits, transport errors, server errors), linear backoff, and configurable callbacks for observability.

Usage

LangEx.LLM.Resilient.chat(LangEx.LLM.Anthropic, messages,
  model: "claude-opus-4-6",
  tools: tools,
  max_retries: 3,
  retry_base_ms: 3_000
)

Options

All options not listed below are forwarded to the underlying provider.

  • :max_retries — maximum retry attempts (default 3)
  • :retry_base_ms — base delay between retries in ms (default 3_000); actual delay is base * (attempt + 1) (linear backoff)
  • :retryable?fn(error_reason) -> boolean() to classify retryable errors (default: 429, transport errors, 5xx)
  • :on_successfn(attempt, duration_ms, ai, usage) -> any()
  • :on_retryfn(attempt, duration_ms, wait_ms, reason) -> any()
  • :on_errorfn(attempt, duration_ms, reason) -> any()
  • :fallbackfn() -> Message.AI.t() called on final failure; when nil, the error propagates as {:error, reason}

Summary

Functions

Call the provider with automatic retries. Returns {:ok, %Message.AI{}}.

Like chat/3 but returns {:ok, %Message.AI{}, usage_map} with token counts and :duration_ms.

Default retryable error classifier.

Format an LLM error reason into a human-readable string.

Functions

chat(provider, messages, opts)

Call the provider with automatic retries. Returns {:ok, %Message.AI{}}.

On final failure, returns the fallback message if configured, or {:error, reason} if no fallback is set.

chat_with_usage(provider, messages, opts)

Like chat/3 but returns {:ok, %Message.AI{}, usage_map} with token counts and :duration_ms.

default_retryable?(arg1)

@spec default_retryable?(term()) :: boolean()

Default retryable error classifier.

Returns true for:

format_error(other)

@spec format_error(term()) :: String.t()

Format an LLM error reason into a human-readable string.