packkit/stream

Streaming helpers for codec families.

packkit/stream exposes opaque decoder states that buffer input chunks and run a one-shot decode at finish time. The API matches the streaming shape the spec calls for - new_*_decoder, push, finish - so callers can already wire incremental pipelines even while the underlying codec decoders are still eager.

Relationship to packkit/codec.Codec. The streaming constructors (new_gzip_encoder, new_zstd_decoder, …) do NOT accept a packkit/codec.Codec value, so the per-codec Level / preset Dictionary settings carried by a Codec are not threaded through this API; every stream uses the codec’s default encode strategy. This is intentional: a Codec is the config object packkit.compress / packkit.decompress (the one-shot facade) consumes, while packkit/stream only ever invokes the codec module’s bare encode/1 / decode/1 entry points. Callers that need a non-default level or a preset dictionary should call the per-codec module directly (gzip.encode_with_header, zlib.encode_with_dictionary, deflate.encode_dynamic, …) and drive the chunking themselves until the streaming API grows codec-typed constructors. The Limits value, by contrast, IS honoured incrementally — see [push] / [push_encoder].

Types

Decoder

opaque </>

Opaque incremental decoder state. The wrapped codec selector is kept private; callers should construct one of the new_*_decoder values and feed it through push/finish.

buffered_bytes is tracked so push can enforce max_input_bytes incrementally — a hostile or buggy producer can no longer stream arbitrarily many chunks into the decoder before the budget check fires at finish time.

pub opaque type Decoder

Encoder

opaque </>

Opaque incremental encoder state. Mirrors Decoder: callers buffer plaintext chunks via push_encoder and pay the actual compress cost once at finish_encoder time. Buffering lets the API stay symmetric with Decoder even though the underlying codecs are still eager — when packkit grows true streaming encoders the public surface won’t have to change.

buffered_bytes is tracked so push_encoder can enforce max_input_bytes incrementally against the input stream, matching the decoder-side limit semantics.

pub opaque type Encoder

Values

decode_chunks

</>

pub fn decode_chunks(
  decoder decoder: Decoder,
  chunks chunks: List(BitArray),
) -> Result(BitArray, error.CodecError)

Convenience helper that pushes every chunk through the decoder in order and returns the final decoded payload. Surfaces the same typed CodecLimitExceeded push would, so a long sequence of chunks cannot silently overrun the input budget.

encode_chunks

</>

pub fn encode_chunks(
  encoder encoder: Encoder,
  chunks chunks: List(BitArray),
) -> Result(BitArray, error.CodecError)

Convenience helper that pushes every plaintext chunk through the encoder in order and returns the final compressed payload. Mirrors decode_chunks so callers can drive both directions with the same shape (list of input chunks → single output buffer).

encoder_with_limits

</>

pub fn encoder_with_limits(
  encoder: Encoder,
  limits limits: limit.Limits,
) -> Encoder

Replace the limits used by an incremental encoder.

finish

</>

pub fn finish(
  decoder: Decoder,
) -> Result(BitArray, error.CodecError)

Finalize the decoder and return the full decoded payload.

finish_encoder

</>

pub fn finish_encoder(
  encoder: Encoder,
) -> Result(BitArray, error.CodecError)

Finalize the encoder and return the compressed payload. Each codec is invoked through its default encode/1 form; advanced per-codec options (gzip header metadata, deflate stored blocks, lz4 content size, bzip2 level) are not exposed through the streaming wrapper and remain accessible via the per-codec module.

new_brotli_decoder

</>

pub fn new_brotli_decoder() -> Decoder

Start a new incremental brotli decoder using the default limits.

new_brotli_encoder

</>

pub fn new_brotli_encoder() -> Encoder

Start a new incremental brotli encoder using the default limits.

new_bzip2_decoder

</>

pub fn new_bzip2_decoder() -> Decoder

Start a new incremental bzip2 decoder using the default limits.

new_bzip2_encoder

</>

pub fn new_bzip2_encoder() -> Encoder

Start a new incremental bzip2 encoder using the default limits.

new_deflate_decoder

</>

pub fn new_deflate_decoder() -> Decoder

Start a new incremental DEFLATE decoder using the default limits.

new_deflate_encoder

</>

pub fn new_deflate_encoder() -> Encoder

Start a new incremental DEFLATE encoder using the default limits.

new_gzip_decoder

</>

pub fn new_gzip_decoder() -> Decoder

Start a new incremental gzip decoder using the default limits.

new_gzip_encoder

</>

pub fn new_gzip_encoder() -> Encoder

Start a new incremental gzip encoder using the default limits. The output uses gzip.default_header() — callers that need per-stream metadata should keep using gzip.encode directly until the streaming API grows an explicit header constructor.

new_lz4_decoder

</>

pub fn new_lz4_decoder() -> Decoder

Start a new incremental LZ4 frame decoder using the default limits.

new_lz4_encoder

</>

pub fn new_lz4_encoder() -> Encoder

Start a new incremental LZ4 frame encoder using the default limits.

new_lzw_decoder

</>

pub fn new_lzw_decoder() -> Decoder

Start a new incremental Unix .Z (LZW) decoder using the default limits.

new_lzw_encoder

</>

pub fn new_lzw_encoder() -> Encoder

Start a new incremental Unix .Z (LZW) encoder using the default limits.

new_snappy_decoder

</>

pub fn new_snappy_decoder() -> Decoder

Start a new incremental Snappy (framed) decoder using the default limits. See [packkit/snappy] for the raw-block variants.

new_snappy_encoder

</>

pub fn new_snappy_encoder() -> Encoder

Start a new incremental Snappy (framed) encoder using the default limits.

new_xz_decoder

</>

pub fn new_xz_decoder() -> Decoder

Start a new incremental xz decoder using the default limits.

new_xz_encoder

</>

pub fn new_xz_encoder() -> Encoder

Start a new incremental xz encoder using the default limits.

new_zlib_decoder

</>

pub fn new_zlib_decoder() -> Decoder

Start a new incremental zlib decoder using the default limits.

new_zlib_encoder

</>

pub fn new_zlib_encoder() -> Encoder

Start a new incremental zlib encoder using the default limits.

new_zstd_decoder

</>

pub fn new_zstd_decoder() -> Decoder

Start a new incremental zstd decoder using the default limits.

new_zstd_encoder

</>

pub fn new_zstd_encoder() -> Encoder

Start a new incremental zstd encoder using the default limits.

push

</>

pub fn push(
  decoder: Decoder,
  chunk: BitArray,
) -> Result(Decoder, error.CodecError)

Append a chunk of input bytes to the decoder, enforcing max_input_bytes incrementally. Returns a typed CodecLimitExceeded if the running buffered byte count would exceed the configured limit.

push_encoder

</>

pub fn push_encoder(
  encoder: Encoder,
  chunk: BitArray,
) -> Result(Encoder, error.CodecError)

Append a chunk of plaintext bytes to the encoder, enforcing max_input_bytes incrementally so an unbounded producer can’t trigger an unbounded encode allocation at finish_encoder time.

with_limits

</>

pub fn with_limits(
  decoder: Decoder,
  limits limits: limit.Limits,
) -> Decoder

Replace the limits used by an incremental decoder.