ExLLM.StreamRecovery (ex_llm v0.5.0)

View Source

Streaming error recovery and resumption support for ExLLM.

This module provides functionality to recover from interrupted streaming responses, allowing continuation from where the stream was interrupted.

Features

  • Automatic saving of partial responses during streaming
  • Detection of recoverable vs non-recoverable errors
  • Multiple resumption strategies
  • Configurable storage backends
  • Automatic cleanup of old partial responses

Usage

# Enable recovery when streaming
{:ok, stream} = ExLLM.stream_chat(:anthropic, messages,
  recovery: [
    enabled: true,
    strategy: :paragraph,
    storage: :memory
  ]
)

# If stream is interrupted, resume it
{:ok, resumed_stream} = ExLLM.StreamRecovery.resume_stream(recovery_id)

Summary

Functions

Returns a specification to start this module under a supervisor.

Cleans up old partial responses.

Clears a partial response from memory.

Marks a stream as completed (no recovery needed).

Gets the partial response (chunks) for a recovery ID.

Initializes a recoverable stream.

Lists all recoverable streams.

Records a chunk for a recoverable stream.

Records an error for potential recovery.

Checks if an error is recoverable.

Attempts to resume a previously interrupted stream.

Starts the StreamRecovery GenServer.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

cleanup_old_responses(ttl \\ 1_800_000)

Cleans up old partial responses.

clear_partial_response(recovery_id)

Clears a partial response from memory.

complete_stream(recovery_id)

Marks a stream as completed (no recovery needed).

get_partial_response(recovery_id)

Gets the partial response (chunks) for a recovery ID.

init_recovery(provider, messages, options)

Initializes a recoverable stream.

list_recoverable_streams()

Lists all recoverable streams.

record_chunk(recovery_id, chunk)

Records a chunk for a recoverable stream.

record_error(recovery_id, error)

Records an error for potential recovery.

recoverable_error?(arg1)

Checks if an error is recoverable.

resume_stream(recovery_id, opts \\ [])

Attempts to resume a previously interrupted stream.

Strategies

  • :exact - Continue from exact cutoff point
  • :paragraph - Regenerate from last complete paragraph
  • :summarize - Summarize received content and continue

start_link(opts \\ [])

Starts the StreamRecovery GenServer.