ExLLM.Providers.Shared.StreamingBehavior behaviour (ex_llm v0.8.1)
View SourceShared streaming behavior and utilities for ExLLM adapters.
Provides common patterns for handling Server-Sent Events (SSE) streaming responses from LLM APIs. This module can be used as a behavior or just for its utility functions.
Summary
Callbacks
Callback for parsing a streaming chunk into an ExLLM StreamChunk.
Functions
Accumulate streaming chunks into a complete response.
Create a stream chunk for function calls.
Create a stream chunk for text content.
Common streaming response handler that works with HTTPoison async responses.
Parse Server-Sent Events from a data stream.
Callbacks
@callback parse_stream_chunk(String.t()) :: {:ok, ExLLM.Types.StreamChunk.t() | :done} | {:error, term()}
Callback for parsing a streaming chunk into an ExLLM StreamChunk.
Each provider has a different chunk format, so adapters must implement this.
Functions
@spec accumulate_chunks([ExLLM.Types.StreamChunk.t()]) :: map()
Accumulate streaming chunks into a complete response.
Useful for collecting all chunks before processing.
@spec create_function_chunk(String.t(), String.t(), keyword()) :: ExLLM.Types.StreamChunk.t()
Create a stream chunk for function calls.
@spec create_text_chunk( String.t(), keyword() ) :: ExLLM.Types.StreamChunk.t()
Create a stream chunk for text content.
@spec handle_stream(reference(), module(), function(), keyword()) :: {:ok, any()} | {:error, term()}
Common streaming response handler that works with HTTPoison async responses.
Options
:timeout
- Stream timeout in milliseconds (default: 5 minutes):buffer
- Initial buffer content (default: ""):on_error
- Error callback function
Examples
StreamingBehavior.handle_stream(ref, MyAdapter, fn chunk ->
# Process each chunk
send(self(), {:chunk, chunk})
end)
Parse Server-Sent Events from a data stream.
Returns a list of parsed events and the remaining buffer.