ReqLLM.Provider.InProcessStream (ReqLLM v1.26.0)

Describes a provider-owned in-process stream of canonical chunks.

Providers can return an enumerable directly from ReqLLM.Provider.attach_in_process_stream/3. Use this struct when the provider also needs a callback when ReqLLM cancels the stream or a timeout stops it.

stream =
  Stream.map(events, fn event ->
    ReqLLM.StreamChunk.text(event.text)
  end)

ReqLLM.Provider.InProcessStream.new(stream,
  cancel: fn -> MyProvider.cancel(subscription) end
)

The enumerable must emit ReqLLM.StreamChunk structs. It can emit {:error, reason} to fail the stream.

Summary

Types

t()

Functions

new(stream, opts \\ [])

Creates an in-process stream definition.

Types

t()

@type t() :: %ReqLLM.Provider.InProcessStream{
  cancel: (-> any()) | nil,
  stream: Enumerable.t()
}

Functions

new(stream, opts \\ [])

@spec new(Enumerable.t(), keyword()) :: t()

Creates an in-process stream definition.

The optional :cancel callback runs when ReqLLM cancels the stream or stops it because of a timeout.