Bedrock.ObjectStorage.ChunkReader (bedrock v0.5.3)
View SourceReads transaction chunks from object storage.
ChunkReader provides efficient access to transaction data stored in chunks, supporting:
- Listing chunks in version order (newest first due to inverted keys)
- Seeking to a specific version
- Forward scanning from a version
- Lazy streaming for replay workflows
Replay Workflow
Cold start playback uses the following approach:
- List chunks (lazy stream, fetches pages as needed)
- Find the chunk containing the target version
- Seek within that chunk using the directory
- Read forward through successive chunks
Example
reader = ChunkReader.new(backend, "a")
# List all chunks (newest first)
chunks = ChunkReader.list_chunks(reader) |> Enum.to_list()
# Read transactions from a specific version forward
transactions = ChunkReader.read_from_version(reader, target_version)
|> Enum.each(fn {version, data} -> process(version, data) end)Summary
Functions
Finds the chunk containing a specific version.
Gets the latest (highest) version in the shard.
Lists chunk metadata (key and version range) without reading full chunks.
Lists chunk keys in the shard.
Creates a new chunk reader for a shard.
Gets the oldest (lowest) version in the shard.
Reads all transactions from all chunks.
Reads a complete chunk by key.
Reads just the header of a chunk.
Reads all transactions from a specific version forward.
Types
@type t() :: %Bedrock.ObjectStorage.ChunkReader{ backend: Bedrock.ObjectStorage.backend(), shard_tag: String.t() }
@type transaction_data() :: binary()
@type version() :: non_neg_integer()
Functions
Finds the chunk containing a specific version.
Returns the chunk key if found, or nil if no chunk contains that version.
Searches through chunks (newest first) until finding one where
min_version <= target <= max_version.
Gets the latest (highest) version in the shard.
Returns nil if no chunks exist.
@spec list_chunk_metadata( t(), keyword() ) :: Enumerable.t()
Lists chunk metadata (key and version range) without reading full chunks.
Returns a lazy stream of {key, min_version, max_version} tuples.
Requires reading the header of each chunk. By default, header decode failures
raise ChunkReader.ReadError to avoid silent replay gaps.
Options
:on_error-:raise(default) or:skip
@spec list_chunks( t(), keyword() ) :: Enumerable.t()
Lists chunk keys in the shard.
Returns a lazy stream of chunk keys in lexicographic order. Due to inverted version keys, this means newest chunks first.
Options
:limit- Maximum number of chunks to return
@spec new(Bedrock.ObjectStorage.backend(), String.t()) :: t()
Creates a new chunk reader for a shard.
Gets the oldest (lowest) version in the shard.
Note: This requires scanning all chunks, so it's less efficient than latest_version/1.
Returns nil if no chunks exist.
@spec read_all_transactions( t(), keyword() ) :: Enumerable.t()
Reads all transactions from all chunks.
Returns a lazy stream of {version, data} tuples in ascending version order
(reads chunks from oldest to newest).
@spec read_chunk(t(), String.t()) :: {:ok, Bedrock.ObjectStorage.Chunk.t()} | {:error, term()}
Reads a complete chunk by key.
Returns
{:ok, chunk}- Decoded chunk{:error, reason}- Read or decode failed
@spec read_chunk_header(t(), String.t()) :: {:ok, Bedrock.ObjectStorage.Chunk.header()} | {:error, term()}
Reads just the header of a chunk.
Useful for determining version range without reading full chunk.
@spec read_from_version(t(), version(), keyword()) :: Enumerable.t()
Reads all transactions from a specific version forward.
Returns a lazy stream of {version, data} tuples starting from the first
transaction with version >= target_version.
The stream:
- Finds the chunk containing the target version
- Seeks to the first transaction >= target within that chunk
- Continues through subsequent chunks in version order
Options
:limit- Maximum number of transactions to return