ExLLM.Adapters.Shared.StreamingCoordinator (ex_llm v0.5.0)

View Source

Unified streaming coordinator for all LLM adapters.

This module provides a consistent streaming implementation that eliminates code duplication across adapters. It handles:

  • Task management for async streaming
  • SSE (Server-Sent Events) parsing
  • Chunk buffering and processing
  • Error recovery integration
  • Provider-agnostic streaming patterns

Summary

Functions

Create a stream collector function for Req's into option.

Execute the actual streaming request with unified SSE handling.

Parse a single SSE line according to the SSE specification.

Create a simple streaming implementation for adapters.

Start a streaming request with unified handling.

Functions

create_stream_collector(callback, parse_chunk_fn, recovery_id, options)

Create a stream collector function for Req's into option.

execute_stream(url, request, headers, callback, parse_chunk_fn, recovery_id, options)

Execute the actual streaming request with unified SSE handling.

parse_sse_line(line)

Parse a single SSE line according to the SSE specification.

process_stream_data(data, buffer, callback, parse_chunk_fn, recovery_id, chunk_count, options)

Process streaming data with unified SSE parsing.

simple_stream(params)

Create a simple streaming implementation for adapters.

This is a high-level function that adapters can use to implement streaming with minimal boilerplate.

Example

def stream_chat(messages, options, callback) do
  base_url = "https://api.openai.com/v1"
  api_key = "your-api-key"

  StreamingCoordinator.simple_stream(
    url: "#{base_url}/chat/completions",
    request: build_request(messages, options),
    headers: build_headers(api_key),
    callback: callback,
    parse_chunk: &parse_openai_chunk/1,
    options: options
  )
end

start_stream(url, request, headers, callback, options \\ [])

Start a streaming request with unified handling.

Options

  • :parse_chunk_fn - Function to parse provider-specific chunks (required)
  • :recovery_id - Optional ID for stream recovery
  • :timeout - Stream timeout in milliseconds (default: 5 minutes)
  • :on_error - Error callback function