ExStorageService.Storage.Pack (ex_storage_service v0.6.3)

Copy Markdown View Source

Immutable, content-addressed pack files for cold blobs.

A pack is an uncompressed concatenation of blob contents at {data_root}/cas/packs/pack-{sha256-of-pack-bytes}.pack, so a packed blob is served with send_file(path, offset, size) — zero-copy, exact Content-Length, and Range = pack_offset + range_offset. CAS identity is preserved: blobs stay addressed by their SHA-256; blob:sha256:{hash} metadata carries state: :packed and pack: %{hash:, offset:}.

The index lives twice: a pack:{pack_hash} Concord record and a JSON .idx sidecar for repair. Packs are never mutated; reclaiming dead entries (repack) is a future follow-up.

Crash-safe write order: tmp pack → rename → sidecar → pack record → per-blob metadata. The loose blob is retained as a fallback until a later Packer cleanup pass, after readers that resolved its old location have had time to finish.

Summary

Functions

Location of a packed blob: {:ok, {pack_path, offset, size}}.

Pack the given loose blobs. Skips hashes whose loose file is missing or whose metadata is already :packed. Returns the new pack's hash and the number of blobs packed.

Read a packed blob's bytes.

Functions

locate(hash)

Location of a packed blob: {:ok, {pack_path, offset, size}}.

pack_blobs(hashes, opts \\ [])

Pack the given loose blobs. Skips hashes whose loose file is missing or whose metadata is already :packed. Returns the new pack's hash and the number of blobs packed.

pack_path(pack_hash)

read(hash)

Read a packed blob's bytes.