Nous.HTTP.StreamBackend.Req (nous v0.15.6)

View Source

Nous.HTTP.StreamBackend implementation backed by Req (Finch underneath).

Default streaming backend. Drives Req.post/1 with the :into callback so chunks are pushed into a Task, which forwards them to the consuming Stream.resource via send/2.

Backpressure

Req's :into callback runs in the spawned Task. Forwarding to the consumer process is send/2, so a fast producer + slow consumer can grow the consumer's mailbox unboundedly. This is acceptable for typical LLM workloads where token-generation rate is the bottleneck and consumers are parsing-bound (parsing throttles naturally).

Callers whose downstream consumers can block per chunk (LiveView fan-out under load, persistence-on-every-chunk, slow IO) should use Nous.HTTP.StreamBackend.Hackney instead, which provides strict pull-based backpressure via :hackney's {:async, :once} mode.

TLS verification

Req's defaults handle TLS verification via Mint/Finch (system CAs with peer verification). No additional configuration needed.