Bedrock.ObjectStorage.Keys (bedrock v0.5.3)

Key formatting helpers for object storage.

Object stores list keys in ascending lexicographic order. To efficiently retrieve the newest objects first, we use inverted version numbers in keys:

inverted = (2^64 - 1) - version

This ensures that newer versions (higher numbers) sort first.

Version numbers are encoded in base36 (0-9, a-z) for compact filenames. A 64-bit integer requires 13 base36 characters.

Path Structure

Paths are relative to the object storage root, which is already cluster-scoped:

Cluster state: state
Chunks: c/{shard}/{version}
Snapshots: s/{shard}/{version}

Summary

Functions

chunk_path(shard_tag, highest_version)

Builds a chunk path for a shard.

chunks_prefix(shard_tag)

Builds a chunks prefix for listing all chunks in a shard.

extract_version(key)

Extracts the version from a chunk or snapshot key.

format_inverted_version(inverted)

Formats an inverted version as a base36-encoded string for lexicographic sorting.

invert_version(version)

Converts a version to an inverted version for sorting.

key_to_version(key)

Parses a key component back to the original version.

parse_inverted_version(str)

Parses a base36-encoded inverted version string back to an integer.

parse_shard_tag(tag)

Parses a shard tag back to an integer shard ID.

restore_version(inverted)

Converts an inverted version back to the original version.

shard_tag(shard_id)

Formats a shard ID as a base36 string for use in paths.

snapshot_path(shard_tag, version)

Builds a snapshot path for a shard.

snapshots_prefix(shard_tag)

Builds a snapshots prefix for listing all snapshots in a shard.

version_to_key(version)

Formats a version as an inverted, base36-encoded string key component.

Functions

chunk_path(shard_tag, highest_version)

@spec chunk_path(String.t(), non_neg_integer()) :: String.t()

Builds a chunk path for a shard.

Examples

iex> Keys.chunk_path("a", 1000)
"c/a/3w5e11264sg0n"

chunks_prefix(shard_tag)

@spec chunks_prefix(String.t()) :: String.t()

Builds a chunks prefix for listing all chunks in a shard.

Examples

iex> Keys.chunks_prefix("a")
"c/a/"

extract_version(key)

@spec extract_version(String.t()) ::
  {:ok, non_neg_integer()} | {:error, :invalid_format}

Extracts the version from a chunk or snapshot key.

Examples

iex> Keys.extract_version("c/a/3w5e11264sg0n")
{:ok, 1000}

iex> Keys.extract_version("invalid!")
{:error, :invalid_format}

format_inverted_version(inverted)

@spec format_inverted_version(non_neg_integer()) :: String.t()

Formats an inverted version as a base36-encoded string for lexicographic sorting.

The string is 13 characters wide, zero-padded.

Examples

iex> Keys.format_inverted_version(0)
"0000000000000"

iex> Keys.format_inverted_version(1000)
"00000000000rs"

invert_version(version)

@spec invert_version(non_neg_integer()) :: non_neg_integer()

Converts a version to an inverted version for sorting.

Inverted versions sort in descending order (newest first).

Examples

iex> Keys.invert_version(0)
18446744073709551615

iex> Keys.invert_version(18446744073709551615)
0

iex> Keys.invert_version(1000)
18446744073709550615

key_to_version(key)

@spec key_to_version(String.t()) ::
  {:ok, non_neg_integer()} | {:error, :invalid_format}

Parses a key component back to the original version.

Combines parse_inverted_version/1 and restore_version/1.

Examples

iex> Keys.key_to_version("3w5e11264sgsf")
{:ok, 0}

iex> Keys.key_to_version("0000000000000")
{:ok, 18446744073709551615}

parse_inverted_version(str)

@spec parse_inverted_version(String.t()) ::
  {:ok, non_neg_integer()} | {:error, :invalid_format}

Parses a base36-encoded inverted version string back to an integer.

Examples

iex> Keys.parse_inverted_version("0000000000000")
{:ok, 0}

iex> Keys.parse_inverted_version("00000000000rs")
{:ok, 1000}

iex> Keys.parse_inverted_version("invalid!")
{:error, :invalid_format}

parse_shard_tag(tag)

@spec parse_shard_tag(String.t()) ::
  {:ok, non_neg_integer()} | {:error, :invalid_format}

Parses a shard tag back to an integer shard ID.

Examples

iex> Keys.parse_shard_tag("0")
{:ok, 0}

iex> Keys.parse_shard_tag("z")
{:ok, 35}

iex> Keys.parse_shard_tag("rs")
{:ok, 1000}

restore_version(inverted)

@spec restore_version(non_neg_integer()) :: non_neg_integer()

Converts an inverted version back to the original version.

Examples

iex> Keys.restore_version(18446744073709551615)
0

iex> Keys.restore_version(0)
18446744073709551615

shard_tag(shard_id)

@spec shard_tag(non_neg_integer()) :: String.t()

Formats a shard ID as a base36 string for use in paths.

Examples

iex> Keys.shard_tag(0)
"0"

iex> Keys.shard_tag(35)
"z"

iex> Keys.shard_tag(1000)
"rs"

snapshot_path(shard_tag, version)

@spec snapshot_path(String.t(), non_neg_integer()) :: String.t()

Builds a snapshot path for a shard.

Examples

iex> Keys.snapshot_path("a", 1000)
"s/a/3w5e11264sg0n"

snapshots_prefix(shard_tag)

@spec snapshots_prefix(String.t()) :: String.t()

Builds a snapshots prefix for listing all snapshots in a shard.

Examples

iex> Keys.snapshots_prefix("a")
"s/a/"

version_to_key(version)

@spec version_to_key(non_neg_integer()) :: String.t()

Formats a version as an inverted, base36-encoded string key component.

Combines invert_version/1 and format_inverted_version/1.

Examples

iex> Keys.version_to_key(0)
"3w5e11264sgsf"

iex> Keys.version_to_key(18446744073709551615)
"0000000000000"