macula_manifest (macula v13.3.0)
View SourceFixed-size chunking, Merkle-root computation, and manifest construction for content larger than one storage block.
Every SDK builds the same manifest for the same bytes, bit for bit: the same MCID format, default chunk size (256 KiB), Merkle fold, canonical-CBOR MCID derivation and wire shape. A sharer serves this manifest and a fetcher from another SDK verifies it (macula_content_store, macula_content_fetch, D27). It hashes with SHA-384 through OTP crypto and encodes with the deterministic CBOR encoder (macula_record_cbor).
MCID format (50 bytes): <<Tag:8, Codec:8, Hash:48/binary>>. The tag names the hash; the post-quantum format has only tag 2, SHA-384 (D24). ?CODEC_RAW (16#55) addresses a single chunk (or a whole blob that fits in one chunk, see macula_content_store). ?CODEC_MANIFEST (16#56) addresses a manifest describing many chunks.
Summary
Functions
The MCID a chunk at Index is served and fetched under: tag 2, codec raw, and the chunk's SHA-384 hash. A sharer and a fetcher derive it independently, and a fetched chunk is checked against it (macula_content_fetch).
Split Data into fixed-size chunks and build its manifest. Returns the manifest and the chunk bytes in order (index 0 first), which a sharer keeps and serves (macula_content_store). Options
Read a manifest as it arrives in a stream DATA body from a sharer (macula_content_fetch): the map as the stream codec decoded it, not the canonical {text,_} record shape. Robust to atom keys, to binary-string keys (mirroring macula_record:payload_field/2), and to {text, Bin} keys: the frame decoder resolves a key to an atom only when that atom already exists, so in a node that has not yet loaded this module the field names arrive as text. A name or hash algorithm sent as text is read as its binary value. A manifest without an mcid, whose chunks are not a list of maps, or that does not name sha384 as its hash algorithm, is {error, invalid_manifest}. So is one that does not describe whole content (whole/1), before any caller sizes or counts anything by it.
Verify reassembled Data against Manifest: size, then a fresh Merkle root over Data re-chunked the same way. A manifest whose chunk size is not a positive integer is {error, invalid_manifest}, so the re-chunking always ends.
Check that Manifest describes the content Mcid names: the MCID recomputed from the manifest's canonical fields (name, size, chunk_size, chunk_count, hash_algorithm and root_hash) must equal Mcid. The manifest's own mcid field is not consulted; a sender can put anything there. A manifest missing a canonical field, whose name is not valid UTF-8 text, or whose hash algorithm is not one this module knows, does not describe Mcid either.
Types
-type algorithm() :: sha384.
-type chunk_info() :: #{index := non_neg_integer(), offset := non_neg_integer(), size := pos_integer(), hash := binary()}.
-type manifest() :: #{mcid := mcid(), version := pos_integer(), name := binary(), size := non_neg_integer(), created := non_neg_integer(), chunk_size := pos_integer(), chunk_count := non_neg_integer(), hash_algorithm := algorithm(), root_hash := binary(), chunks := [chunk_info()]}.
-type mcid() :: <<_:400>>.
Functions
-spec chunk_mcid(manifest(), non_neg_integer()) -> {ok, mcid()} | {error, invalid_index}.
The MCID a chunk at Index is served and fetched under: tag 2, codec raw, and the chunk's SHA-384 hash. A sharer and a fetcher derive it independently, and a fetched chunk is checked against it (macula_content_fetch).
Split Data into fixed-size chunks and build its manifest. Returns the manifest and the chunk bytes in order (index 0 first), which a sharer keeps and serves (macula_content_store). Options:
name— content name (default<<"unnamed">>)chunk_size— bytes per chunk, a positive integer (defaultdefault_chunk_size/0)hash_algorithm:sha384, the only one and the default
-spec default_chunk_size() -> pos_integer().
Read a manifest as it arrives in a stream DATA body from a sharer (macula_content_fetch): the map as the stream codec decoded it, not the canonical {text,_} record shape. Robust to atom keys, to binary-string keys (mirroring macula_record:payload_field/2), and to {text, Bin} keys: the frame decoder resolves a key to an atom only when that atom already exists, so in a node that has not yet loaded this module the field names arrive as text. A name or hash algorithm sent as text is read as its binary value. A manifest without an mcid, whose chunks are not a list of maps, or that does not name sha384 as its hash algorithm, is {error, invalid_manifest}. So is one that does not describe whole content (whole/1), before any caller sizes or counts anything by it.
-spec verify(manifest(), binary()) -> ok | {error, size_mismatch | root_hash_mismatch | invalid_manifest}.
Verify reassembled Data against Manifest: size, then a fresh Merkle root over Data re-chunked the same way. A manifest whose chunk size is not a positive integer is {error, invalid_manifest}, so the re-chunking always ends.
Check that Manifest describes the content Mcid names: the MCID recomputed from the manifest's canonical fields (name, size, chunk_size, chunk_count, hash_algorithm and root_hash) must equal Mcid. The manifest's own mcid field is not consulted; a sender can put anything there. A manifest missing a canonical field, whose name is not valid UTF-8 text, or whose hash algorithm is not one this module knows, does not describe Mcid either.