ExLLM.StreamRecovery (ex_llm v0.5.0)
View SourceStreaming error recovery and resumption support for ExLLM.
This module provides functionality to recover from interrupted streaming responses, allowing continuation from where the stream was interrupted.
Features
- Automatic saving of partial responses during streaming
- Detection of recoverable vs non-recoverable errors
- Multiple resumption strategies
- Configurable storage backends
- Automatic cleanup of old partial responses
Usage
# Enable recovery when streaming
{:ok, stream} = ExLLM.stream_chat(:anthropic, messages,
recovery: [
enabled: true,
strategy: :paragraph,
storage: :memory
]
)
# If stream is interrupted, resume it
{:ok, resumed_stream} = ExLLM.StreamRecovery.resume_stream(recovery_id)
Summary
Functions
Returns a specification to start this module under a supervisor.
Cleans up old partial responses.
Clears a partial response from memory.
Marks a stream as completed (no recovery needed).
Gets the partial response (chunks) for a recovery ID.
Initializes a recoverable stream.
Lists all recoverable streams.
Records a chunk for a recoverable stream.
Records an error for potential recovery.
Checks if an error is recoverable.
Attempts to resume a previously interrupted stream.
Starts the StreamRecovery GenServer.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor
.
Cleans up old partial responses.
Clears a partial response from memory.
Marks a stream as completed (no recovery needed).
Gets the partial response (chunks) for a recovery ID.
Initializes a recoverable stream.
Lists all recoverable streams.
Records a chunk for a recoverable stream.
Records an error for potential recovery.
Checks if an error is recoverable.
Attempts to resume a previously interrupted stream.
Strategies
:exact
- Continue from exact cutoff point:paragraph
- Regenerate from last complete paragraph:summarize
- Summarize received content and continue
Starts the StreamRecovery GenServer.