ExAgent.Services.Streaming (ExAgent v0.2.0)

Copy Markdown View Source

Shared Server-Sent Events (SSE) streaming helper for LLM providers.

Wraps a Req request made with into: :self in a lazy Stream that yields text chunks. SSE framing (buffering split frames, data: prefixes, [DONE] termination) lives here so provider services only supply a per-provider parser that extracts the text delta from a decoded frame.

Summary

Functions

Streams an SSE response as a lazy enumerable of text chunks.

Types

chunk_parser()

@type chunk_parser() :: (map() -> String.t() | nil)

Functions

stream(req, req_opts, chunk_parser)

@spec stream(Req.Request.t(), keyword(), chunk_parser()) :: Enumerable.t()

Streams an SSE response as a lazy enumerable of text chunks.

req_opts are passed to Req.post/2 (e.g. url:, json:, receive_timeout:); into: :self is added automatically. chunk_parser receives each decoded SSE frame (a map) and returns the text to emit, or nil to skip it.

Raises ExAgent.StreamError (when consumed) on a non-200 response.