LlmEx.StreamProcessor (LlmEx v0.1.0)

View Source

Shared functionality for processing streaming responses from LLM providers. Handles common patterns for streaming responses and managing events.

Summary

Functions

Process JSON deltas and accumulate them for tools that send partial JSON.

Handle line-by-line streaming with SSE (Server-Sent Events) format.

Parse a single SSE (Server-Sent Events) line into event type and data.

Process a streaming response with a custom handler function.

Functions

accumulate_json_delta(partial_json, tool_key, get_fn, put_fn, field \\ :input)

Process JSON deltas and accumulate them for tools that send partial JSON.

Parameters

  • partial_json - The partial JSON string to process
  • tool_key - Unique key for the tool (usually {message_id, index})
  • get_fn - Function to get the current state
  • put_fn - Function to store the updated state
  • field - Field to update (default: :input)

Returns

  • The updated state

handle_sse_stream(data, state, process_event_fn)

Handle line-by-line streaming with SSE (Server-Sent Events) format.

Parameters

  • data - The data chunk to process
  • state - Current state including buffer and processing details
  • process_event_fn - Function to process each event

parse_sse_line(line)

Parse a single SSE (Server-Sent Events) line into event type and data.

Parameters

  • line - The SSE line to parse

Returns

  • {:event, event_type, parsed_data} - For event lines with JSON data
  • {:data, data} - For data lines
  • :unknown - For other lines

stream_processor(handler_func, message_id, pid, opts \\ [])

Process a streaming response with a custom handler function.

Parameters

  • handler_func - Function to process each data chunk
  • message_id - ID of the message being processed
  • pid - Process to send events to
  • opts - Additional options

Returns

  • A function that can be used with Req.post! :into option