LLM.Stream (LLM v0.1.0)

Stream handling for LLM API responses.

Provides functions to:

Start a streaming request
Receive individual chunks
Collect all chunks into a final response
Auto-execute tool calls and loop

Usage

{:ok, stream} = LLM.stream("Hello", provider: :openai, model: "gpt-4")
{:ok, response} = LLM.Stream.collect(stream, on_chunk: &IO.write/1)

Summary

Types

chunk_type()

t()

Functions

collect(stream, opts \\ [])

Collect all chunks into a final response, executing tool calls automatically.

next(stream)

Receive the next chunk from the stream.

start(context, opts)

Start a streaming request. Returns {:ok, stream} or {:error, reason}.

Types

chunk_type()

@type chunk_type() ::
  LLM.Stream.Chunk.t()
  | LLM.Stream.ToolCall.t()
  | LLM.Stream.Thinking.t()
  | LLM.Stream.Stop.t()
  | LLM.Stream.Error.t()

t()

@type t() :: %LLM.Stream{
  buffer: String.t(),
  context: LLM.Context.t(),
  dialect: module(),
  done: boolean(),
  headers: [{String.t(), String.t()}],
  opts: keyword(),
  provider: map(),
  ref: Req.Response.t(),
  rounds: non_neg_integer(),
  url: String.t()
}

Functions

collect(stream, opts \\ [])

@spec collect(
  t(),
  keyword()
) :: {:ok, LLM.Response.t()} | {:error, term()}

Collect all chunks into a final response, executing tool calls automatically.

Options

:auto_tools — auto-execute tool calls (default: true)
:max_rounds — max tool call rounds (default: 10)
:on_chunk — callback for each chunk, fn chunk -> ... end

next(stream)

@spec next(t()) :: {:ok, [chunk_type()], t()} | {:halt, t()} | {:error, term()}

Receive the next chunk from the stream.

Returns:

{:ok, [chunk], stream} — one or more chunks received
{:halt, stream} — stream ended
{:error, reason} — error occurred

start(context, opts)

@spec start(
  LLM.Context.t(),
  keyword()
) :: {:ok, t()} | {:error, term()}

Start a streaming request. Returns {:ok, stream} or {:error, reason}.