Nous.HTTP.StreamBackend behaviour (nous v0.15.7)
View SourceBehaviour for SSE / chunked streaming HTTP backends.
Implemented by Nous.HTTP.StreamBackend.Req (default) and
Nous.HTTP.StreamBackend.Hackney. Selection mirrors the non-streaming
Nous.HTTP.Backend resolution chain (per-call → env var → app config →
default). See Nous.Providers.HTTP.stream/4 for the resolution order.
Pick a backend three ways, highest precedence first:
# 1. Per-call opt
Nous.Providers.HTTP.stream(url, body, headers,
stream_backend: Nous.HTTP.StreamBackend.Hackney)
# 2. Environment variable (req | hackney | "MyApp.MyStreamBackend")
export NOUS_HTTP_STREAM_BACKEND=hackney
# 3. Application config
config :nous, :http_stream_backend, Nous.HTTP.StreamBackend.HackneyDefault: Nous.HTTP.StreamBackend.Req.
When to pick which
Nous.HTTP.StreamBackend.Req— one HTTP stack across streaming and non-streaming, simpler dependency story. Right default for most apps. Backpressure is bounded by parsing speed, not bystream_next/1pacing — a fast LLM + slow consumer can grow the consumer's mailbox. Acceptable for typical LLM workloads where token rate is the bottleneck.Nous.HTTP.StreamBackend.Hackney— strict pull-based backpressure via:hackney's{:async, :once}mode. The consumer paces the producer chunk-by-chunk. Pick this when downstream consumers can block per chunk (LiveView assigns + diff + push under load, persistence-on-every-chunk, slow IO).
Both backends emit the same normalized event stream (parsed JSON maps,
{:stream_done, reason}, {:stream_error, reason}). Switching between
them does not require changes elsewhere.
Custom backends
Implement stream/4 and return {:ok, Enumerable.t()} where the
enumerable emits parsed JSON maps, {:stream_done, reason} tuples, or
{:stream_error, reason} tuples. The stream MUST halt after the first
{:stream_error, _} and after {:stream_done, _}.
Summary
Callbacks
Issue a streaming POST request and return a lazy Enumerable.t() of
parsed events.
Callbacks
@callback stream( url :: String.t(), body :: map(), headers :: [{String.t(), String.t()}], opts :: keyword() ) :: {:ok, Enumerable.t()} | {:error, term()}
Issue a streaming POST request and return a lazy Enumerable.t() of
parsed events.
Options
:timeout— receive timeout in milliseconds (default:180_000):connect_timeout— TCP connect timeout in milliseconds (default:30_000):stream_parser— module implementingparse_buffer/1for non-SSE formats (e.g. JSON-array streams). Defaults to SSE.
Backends MAY accept additional options; unknown options should be ignored.