# Data Formats Potable relies on four foundational formats: - DAG-CBOR (`Potable.DagCbor`) - CID (`Potable.CID`) - varint (`Potable.Varint`) - CAR v1 (`Potable.CAR`) ## DAG-CBOR (`Potable.DagCbor`) Potable encodes data using deterministic DAG-CBOR rules: - map keys must be strings - map keys are sorted lexicographically by UTF-8 bytes - `nil` is encoded as CBOR null (`0xf6`) and is allowed in maps - CID links are encoded as CBOR tag 42 with payload `<<0x00>> <> cid_bytes` - tag-42 payloads without `0x00` prefix are rejected during decode - integers and lengths use canonical CBOR forms ### Supported Encoded Terms - strings - integers (positive and negative) - booleans - `nil` - lists - maps (string keys only) - `%CBOR.Tag{tag: :bytes, value: binary}` for byte strings - `%Potable.CID{}` for IPLD links - floats (note: repository write validation rejects float values by default) ### Example ```elixir map = %{"z" => 1, "a" => 2, "prev" => nil} {:ok, bytes} = Potable.DagCbor.encode(map) {:ok, decoded} = Potable.DagCbor.decode(bytes) ``` `decoded` is `%{"a" => 2, "prev" => nil, "z" => 1}`. ### Constraint Failures Examples that fail: - `%{atom_key: "v"}` (non-string key) - malformed tag-42 payload without identity prefix byte ## CID (`Potable.CID`) CIDs are content identifiers referencing block bytes. Potable supports: - strict parsing for CIDv0 and CIDv1 (`b...`, `z...`, `Qm...`) - creating CIDv1 from bytes (`create_from_bytes/2`) - verifying bytes against CID (`verify_bytes/2`) - CID binary conversion (`to_binary/1`) for base32/base58/CIDv0 ### Creating CID From Bytes ```elixir bytes = Potable.DagCbor.encode!(%{"hello" => "world"}) cid = Potable.CID.create_from_bytes(bytes) # default codec :dag_cbor ``` ### Verifying Content ```elixir :ok = Potable.CID.verify_bytes(cid, bytes) ``` `verify_bytes/2` returns: - `:ok` - `:mismatch` - `{:error, :unsupported_multihash}` - `{:error, :invalid_multihash}` ## Varint (`Potable.Varint`) Varint is used for: - multicodec/multihash parsing - CAR section length prefixes ```elixir encoded = Potable.Varint.encode(300) {:ok, 300, rest} = Potable.Varint.decode(encoded <> "tail") ``` ## CAR v1 (`Potable.CAR`) CAR archives package blocks plus root CID references. Binary layout: 1. header length varint 2. DAG-CBOR header map `%{"roots" => [...], "version" => 1}` 3. repeated block sections: - section length varint - raw binary CID bytes - block bytes ### Export ```elixir {:ok, car} = Potable.CAR.export([root_cid], [{cid1, bytes1}, {cid2, bytes2}]) ``` ### Import ```elixir {:ok, %{roots: roots, blocks: blocks}} = Potable.CAR.import(car) ``` Import validates CID integrity for each block and returns typed errors instead of raising. ### Import Into Store ```elixir {:ok, result} = Potable.CAR.import_into(car, store) # result => %{roots: [...], block_count: n} ```