ExZarr.MetadataCache (ExZarr v1.1.0)

ETS-based cache for Zarr array and group metadata.

Caches metadata to reduce repeated file system or cloud storage reads. Particularly beneficial for cloud storage where metadata reads have significant latency and cost.

Features

ETS-based storage: Fast, concurrent reads
TTL support: Configurable time-to-live for cache entries
Automatic invalidation: Clear cache on metadata writes
Statistics tracking: Monitor cache hit/miss rates
Process-safe: GenServer manages cache lifecycle

Configuration

config :ex_zarr, :metadata_cache,
  enabled: true,
  ttl: 300_000,      # 5 minutes in milliseconds
  max_size: 1000      # Maximum cached entries

Usage

# Start the cache (usually done by Application supervisor)
{:ok, pid} = ExZarr.MetadataCache.start_link()

# Cache metadata
:ok = ExZarr.MetadataCache.put("/path/to/array", metadata)

# Retrieve cached metadata
{:ok, metadata} = ExZarr.MetadataCache.get("/path/to/array")

# Invalidate on write
:ok = ExZarr.MetadataCache.invalidate("/path/to/array")

# Check statistics
%{hits: 100, misses: 10, hit_rate: 0.909} =
  ExZarr.MetadataCache.stats()

Cache Key Format

Cache keys are derived from storage path and type:

"/path/to/array" - Array metadata
"/path/to/group" - Group metadata

Summary

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

clear(name \\ __MODULE__)

Clear all cached metadata.

get(path, name \\ __MODULE__)

Retrieve metadata from cache.

invalidate(path, name \\ __MODULE__)

Invalidate cached metadata for a path.

put(path, metadata, name \\ __MODULE__)

Store metadata in cache.

start_link(opts \\ [])

Start the metadata cache GenServer.

stats(name \\ __MODULE__)

Get cache statistics.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

clear(name \\ MODULE)

@spec clear(atom()) :: :ok

Clear all cached metadata.

Returns

:ok - Cache cleared

Examples

:ok = ExZarr.MetadataCache.clear()

get(path, name \\ MODULE)

@spec get(String.t(), atom()) :: {:ok, term()} | {:error, :not_found}

Retrieve metadata from cache.

Parameters

path - Storage path to the array or group

Returns

{:ok, metadata} - Cache hit, metadata returned
{:error, :not_found} - Cache miss

Examples

case ExZarr.MetadataCache.get("/data/my_array") do
  {:ok, metadata} ->
    # Use cached metadata
    IO.puts("Cache hit!")
  {:error, :not_found} ->
    # Read from storage
    IO.puts("Cache miss, reading from disk")
end

invalidate(path, name \\ MODULE)

@spec invalidate(String.t(), atom()) :: :ok

Invalidate cached metadata for a path.

Should be called whenever metadata is written to storage.

Parameters

path - Storage path to invalidate

Returns

:ok - Entry invalidated (or didn't exist)

Examples

# After writing new metadata
:ok = ExZarr.Storage.write_metadata(backend, metadata)
:ok = ExZarr.MetadataCache.invalidate("/data/my_array")

put(path, metadata, name \\ MODULE)

@spec put(String.t(), term(), atom()) :: :ok

Store metadata in cache.

Parameters

path - Storage path to the array or group
metadata - Metadata to cache (any term)

Returns

:ok - Metadata cached successfully

Examples

metadata = %ExZarr.MetadataV3{...}
:ok = ExZarr.MetadataCache.put("/data/my_array", metadata)

start_link(opts \\ [])

Start the metadata cache GenServer.

Options

:name - Registered name (default: __MODULE__)
:ttl - Time-to-live in milliseconds (default: 300_000)
:max_size - Maximum cache entries (default: 1000)

stats(name \\ MODULE)

@spec stats(atom()) :: map()

Get cache statistics.

Returns

Map with :hits, :misses, :entries, :hit_rate

Examples

stats = ExZarr.MetadataCache.stats()
hit_rate_pct = stats.hit_rate * 100
IO.puts("Hit rate: #{hit_rate_pct}%")
IO.puts("Total entries: #{stats.entries}")