ExLLM.Providers.Shared.StreamingCoordinator (ex_llm v0.8.1)
View SourceUnified streaming coordinator for all LLM adapters.
This module provides a consistent streaming implementation that eliminates code duplication across adapters. It handles:
- Task management for async streaming
- SSE (Server-Sent Events) parsing
- Chunk buffering and processing
- Error recovery integration
- Provider-agnostic streaming patterns
Summary
Functions
Create a stream collector function for Req's into option.
Execute the actual streaming request with unified SSE handling.
Parse a single SSE line according to the SSE specification.
Process streaming data with unified SSE parsing and buffering.
Create a simple streaming implementation for adapters.
Start a streaming request with unified handling.
Functions
Create a stream collector function for Req's into option.
Execute the actual streaming request with unified SSE handling.
Parse a single SSE line according to the SSE specification.
Process streaming data with unified SSE parsing and buffering.
Create a simple streaming implementation for adapters.
This is a high-level function that adapters can use to implement streaming with minimal boilerplate.
Example
def stream_chat(messages, options, callback) do
base_url = "https://api.openai.com/v1"
api_key = "your-api-key"
StreamingCoordinator.simple_stream(
url: "#{base_url}/chat/completions",
request: build_request(messages, options),
headers: build_headers(api_key),
callback: callback,
parse_chunk: &parse_openai_chunk/1,
options: options
)
end
Start a streaming request with unified handling.
Options
:parse_chunk_fn
- Function to parse provider-specific chunks (required):recovery_id
- Optional ID for stream recovery:timeout
- Stream timeout in milliseconds (default: 5 minutes):on_error
- Error callback function:on_metrics
- Metrics callback function (receives streaming metrics):transform_chunk
- Optional function to transform chunks before callback:buffer_chunks
- Buffer size for chunk batching (default: 1):validate_chunk
- Optional function to validate chunks:track_metrics
- Enable detailed metrics tracking (default: false)