Bedrock.ObjectStorage.Snapshot (bedrock v0.5.2)

View Source

Snapshot storage for materialized shard state.

Snapshots capture the complete state of a shard at a specific version, allowing materializers to cold start from a known point rather than replaying all transactions.

Path Structure

Snapshots are stored at: /shards/{tag}/snapshots/{inverted_version} (relative to the object storage root, which is already cluster-scoped)

Using inverted versions ensures listing returns newest snapshots first, making it efficient to find the latest snapshot.

Conditional Writes

Snapshots use conditional writes (put_if_not_exists) to prevent duplicate writes. Since snapshot data is deterministic (same version = same state), concurrent attempts to write the same snapshot are idempotent - the first write wins, subsequent attempts see "already exists" and can safely skip.

Usage

snapshot = Snapshot.new(backend, "a")

# Write a snapshot
:ok = Snapshot.write(snapshot, version, state_binary)

# Read latest snapshot
case Snapshot.read_latest(snapshot) do
  {:ok, version, data} -> load_state(version, data)
  {:error, :not_found} -> start_from_scratch()
end

# Read specific snapshot
{:ok, data} = Snapshot.read(snapshot, version)

Summary

Types

snapshot_data()
t()
version()

Functions

count(snapshot)

Counts the number of snapshots.

delete(snapshot, version)

Deletes a specific snapshot.

delete_older_than(snapshot, min_version_to_keep)

Deletes all snapshots older than the given version.

exists?(snapshot)

Checks if any snapshots exist for this shard.

latest_version(snapshot)

Gets the latest snapshot version without reading the data.

list(snapshot, opts \\ [])

Lists all snapshots in newest-first order.

new(backend, shard_tag)

Creates a new snapshot handler for a shard.

read(snapshot, version)

Reads a specific snapshot by version.

read_latest(snapshot)

Reads the latest (highest version) snapshot.

write(snapshot, version, data)

Writes a snapshot using conditional put.

Types

snapshot_data()

@type snapshot_data() :: iodata()

t()

@type t() :: %Bedrock.ObjectStorage.Snapshot{
  backend: Bedrock.ObjectStorage.backend(),
  shard_tag: String.t()
}

version()

@type version() :: non_neg_integer()

Functions

count(snapshot)

@spec count(t()) :: non_neg_integer()

Counts the number of snapshots.

Note: This reads the full list, so it's not efficient for shards with many snapshots. Use exists?/1 to just check for presence.

delete(snapshot, version)

@spec delete(t(), version()) :: :ok | {:error, term()}

Deletes a specific snapshot.

Deletion is idempotent - deleting a non-existent snapshot succeeds.

Returns

  • :ok - Snapshot deleted (or didn't exist)
  • {:error, reason} - Delete failed

delete_older_than(snapshot, min_version_to_keep)

@spec delete_older_than(t(), version()) :: {:ok, non_neg_integer()} | {:error, term()}

Deletes all snapshots older than the given version.

Useful for cleanup after compaction or when retention policy expires.

Returns

  • {:ok, deleted_count} - Number of snapshots deleted
  • {:error, reason} - Delete failed (partial deletions may have occurred)

exists?(snapshot)

@spec exists?(t()) :: boolean()

Checks if any snapshots exist for this shard.

latest_version(snapshot)

@spec latest_version(t()) :: {:ok, version()} | {:error, :not_found}

Gets the latest snapshot version without reading the data.

Returns

  • {:ok, version} - Latest version
  • {:error, :not_found} - No snapshots exist

list(snapshot, opts \\ [])

@spec list(
  t(),
  keyword()
) :: Enumerable.t()

Lists all snapshots in newest-first order.

Returns a lazy stream of {version, key} tuples.

Options

  • :limit - Maximum number of snapshots to return

new(backend, shard_tag)

@spec new(Bedrock.ObjectStorage.backend(), String.t()) :: t()

Creates a new snapshot handler for a shard.

read(snapshot, version)

@spec read(t(), version()) :: {:ok, snapshot_data()} | {:error, :not_found | term()}

Reads a specific snapshot by version.

Returns

  • {:ok, data} - Snapshot data
  • {:error, :not_found} - Snapshot doesn't exist
  • {:error, reason} - Read failed

read_latest(snapshot)

@spec read_latest(t()) ::
  {:ok, version(), snapshot_data()} | {:error, :not_found | term()}

Reads the latest (highest version) snapshot.

Returns

  • {:ok, version, data} - Latest snapshot found
  • {:error, :not_found} - No snapshots exist
  • {:error, reason} - Read failed

write(snapshot, version, data)

@spec write(t(), version(), snapshot_data()) :: :ok | {:error, term()}

Writes a snapshot using conditional put.

If a snapshot already exists for this version, returns :ok (idempotent). The data can be binary or iodata (list of binaries).

Returns

  • :ok - Snapshot written (or already existed)
  • {:error, reason} - Write failed