ExZarr.Storage.Backend.Filesystem (ExZarr v1.1.0)

View Source

Local filesystem storage backend for Zarr arrays.

Stores chunks and metadata on the local filesystem following the Zarr v2 directory structure specification. Each chunk is stored as a separate file with dot-separated indices.

Directory Structure

/path/to/array/
  .zarray          # JSON metadata file
  0                # Chunk at index {0} (1D)
  0.0              # Chunk at index {0, 0} (2D)
  0.1              # Chunk at index {0, 1}
  1.0              # Chunk at index {1, 0}
  ...

Characteristics

  • Persistent: Data survives process restarts
  • Portable: Standard Zarr format, interoperable with Python zarr
  • Inspectable: Files can be examined with standard tools
  • Hierarchical: Supports nested directory structures
  • Concurrent: Multiple processes can read simultaneously

Configuration

Requires a :path option specifying the directory location:

{:ok, array} = ExZarr.create(
  shape: {1000, 1000},
  dtype: :float64,
  storage: :filesystem,
  path: "/tmp/my_array"
)

Python Interoperability

Arrays created with this backend can be read by Python zarr:

import zarr
z = zarr.open('/tmp/my_array', mode='r')
data = z[:]