Gleanex.NDJSON (Gleanex v0.1.0)

Copy Markdown View Source

Decode a newline-delimited JSON stream.

This is the shape Glean uses for streaming chat. From the description of ChatRequest.stream:

If set, response lines will be streamed one-by-one as they become available. Each will be a ChatResponse, formatted as JSON, and separated by a new line.

Like Gleanex.SSE, this is a plain function over an enumerable of binary chunks and reassembles lines split across chunk boundaries.

Examples

iex> [~s({"a":1}\n{"a":), ~s(2}\n)] |> Gleanex.NDJSON.decode() |> Enum.to_list()
[%{"a" => 1}, %{"a" => 2}]

Summary

Functions

Turn a stream of binary chunks into a stream of decoded terms.

Functions

decode(chunks)

@spec decode(Enumerable.t()) :: Enumerable.t()

Turn a stream of binary chunks into a stream of decoded terms.

Blank lines are skipped. A malformed line raises Gleanex.Error rather than being dropped, so a truncated or corrupted stream cannot look like a short but complete one.