macula_manifest (macula v9.1.0)

View Source

Fixed-size chunking, Merkle-root computation, and manifest construction for content larger than one storage block.

Mirrors macula-station's macula_manifest / macula_content_chunker / macula_content_hasher BYTE-FOR-BYTE: same MCID format, same default chunk size (256 KiB), same Merkle fold, same canonical-CBOR MCID derivation, same manifest wire shape. This is deliberate, not incidental — the SDK puts a manifest via the station's existing (unmodified) _content.put_manifest / _content.get_manifest RPCs, so the two sides must agree on the algorithm bit-for-bit. Both use the same BLAKE3 NIF (macula_blake3_nif, SDK-owned; the station calls it too) and the same deterministic CBOR encoder (macula_record_cbor, SDK-owned; the station's manifest module calls it directly), so this is a faithful client-side port, not a re-derivation.

MCID format (34 bytes): <<Version:8, Codec:8, Hash:32/binary>>. ?CODEC_RAW (16#55) addresses a single chunk (or a whole blob that fits in one chunk — see the module doc on macula:put_content/2). ?CODEC_MANIFEST (16#56) addresses a manifest describing many chunks.

Summary

Types

algorithm/0
chunk_info/0
manifest/0
mcid/0

Functions

chunk_mcid(_, Index, Algorithm)

The MCID a chunk at Index is stored/fetched under. The station derives this same value independently when serving the chunk, so both sides agree on its address without exchanging it.

create(Data)
create(Data, Opts)

Split Data into fixed-size chunks and build its manifest. Returns the manifest and the chunk bytes in order (index 0 first), so a caller can upload each chunk (via _content.put_block) and then the manifest (via _content.put_manifest). Options

default_chunk_size()
from_wire(M)

Read a manifest as it arrives over _content.get_manifest: the station stores + returns the map exactly as its RPC layer decoded it, with no dedicated re-encode/decode round trip on either side — so the shape depends on the general CALL-result codec, not the canonical {text,_} record shape. Robust to atom keys (the happy path — the field names are atoms defined in this module, so binary_to_existing_atom in the frame decoder resolves them) and to binary-string keys (a defensive fallback, mirroring macula_record:payload_field/2).

verify(Manifest, Data)

Verify reassembled Data against Manifest: size, then a fresh Merkle root over Data re-chunked the same way.

Types

algorithm/0

-type algorithm() :: blake3 | sha256.

chunk_info/0

-type chunk_info() ::
          #{index := non_neg_integer(),
            offset := non_neg_integer(),
            size := pos_integer(),
            hash := binary()}.

manifest/0

-type manifest() ::
          #{mcid := mcid(),
            version := pos_integer(),
            name := binary(),
            size := non_neg_integer(),
            created := non_neg_integer(),
            chunk_size := pos_integer(),
            chunk_count := non_neg_integer(),
            hash_algorithm := algorithm(),
            root_hash := binary(),
            chunks := [chunk_info()]}.

mcid/0

-type mcid() :: <<_:272>>.

Functions

chunk_mcid(_, Index, Algorithm)

-spec chunk_mcid(manifest(), non_neg_integer(), algorithm()) -> {ok, mcid()} | {error, invalid_index}.

The MCID a chunk at Index is stored/fetched under. The station derives this same value independently when serving the chunk, so both sides agree on its address without exchanging it.

create(Data)

-spec create(binary()) -> {ok, manifest(), [binary()]}.

create(Data, Opts)

-spec create(binary(), map()) -> {ok, manifest(), [binary()]}.

Split Data into fixed-size chunks and build its manifest. Returns the manifest and the chunk bytes in order (index 0 first), so a caller can upload each chunk (via _content.put_block) and then the manifest (via _content.put_manifest). Options:

  • name — content name (default <<"unnamed">>)
  • chunk_size — bytes per chunk (default default_chunk_size/0)
  • hash_algorithm — blake3 | sha256 (default blake3)

default_chunk_size()

-spec default_chunk_size() -> pos_integer().

from_wire(M)

-spec from_wire(map()) -> {ok, manifest()} | {error, invalid_manifest}.

Read a manifest as it arrives over _content.get_manifest: the station stores + returns the map exactly as its RPC layer decoded it, with no dedicated re-encode/decode round trip on either side — so the shape depends on the general CALL-result codec, not the canonical {text,_} record shape. Robust to atom keys (the happy path — the field names are atoms defined in this module, so binary_to_existing_atom in the frame decoder resolves them) and to binary-string keys (a defensive fallback, mirroring macula_record:payload_field/2).

verify(Manifest, Data)

-spec verify(manifest(), binary()) -> ok | {error, size_mismatch | root_hash_mismatch}.

Verify reassembled Data against Manifest: size, then a fresh Merkle root over Data re-chunked the same way.