ExZarr.Group (ExZarr v1.1.0)

View Source

Hierarchical groups for organizing Zarr arrays.

Groups allow you to organize multiple arrays in a hierarchical structure, similar to directories in a filesystem or groups in HDF5. This is useful for managing related datasets together.

Group Structure

A group contains:

  • Arrays: Named arrays stored within the group
  • Subgroups: Nested groups for hierarchical organization
  • Attributes: Metadata key-value pairs
  • Storage: Backend storage (shared with child arrays)

Filesystem Layout

Groups are represented on disk as directories with .zgroup files:

/data/
  .zgroup              # Group metadata
  measurements/
    .zarray            # Array metadata
    0.0, 0.1, ...      # Array chunks
  experiments/
    .zgroup            # Subgroup metadata
    results/
      .zarray
      0.0, 0.1, ...

Examples

# Create a root group
{:ok, root} = ExZarr.Group.create("/data",
  storage: :filesystem,
  path: "/tmp/zarr_data"
)

# Create arrays in the group
{:ok, temp} = ExZarr.Group.create_array(root, "temperature",
  shape: {1000, 1000},
  chunks: {100, 100},
  dtype: :float32
)

{:ok, pressure} = ExZarr.Group.create_array(root, "pressure",
  shape: {1000, 1000},
  chunks: {100, 100},
  dtype: :float32
)

# Create subgroups for organization
{:ok, exp1} = ExZarr.Group.create_group(root, "experiment_1")
{:ok, results} = ExZarr.Group.create_array(exp1, "results",
  shape: {500, 500},
  chunks: {50, 50}
)

# Add metadata to groups
root = ExZarr.Group.set_attr(root, "description", "Sensor data collection")
root = ExZarr.Group.set_attr(root, "version", "1.0")

Summary

Functions

Creates multiple groups and/or arrays in parallel.

Creates a new group.

Creates a new array within this group.

Creates a subgroup within this group.

Access nested groups/arrays using path notation. Creates intermediate groups on write operations.

Gets an array from the group by name.

Gets an attribute from the group.

Gets a subgroup by name.

Gets an item (array or group) at the specified path with lazy loading.

Lists all arrays in the group.

Lists all subgroups.

Opens an existing group from storage.

Puts an item (array or group) at the specified path.

Removes an item from the group.

Ensures a group exists at the specified path, creating it if necessary.

Sets an attribute on the group.

Generates an ASCII tree visualization of the group hierarchy.

Types

t()

@type t() :: %ExZarr.Group{
  _loaded: MapSet.t(String.t()),
  arrays: %{required(String.t()) => ExZarr.Array.t()},
  attrs: map(),
  groups: %{required(String.t()) => t()},
  path: String.t(),
  storage: ExZarr.Storage.t()
}

Functions

batch_create(parent, items)

@spec batch_create(t(), [{:group | :array, String.t(), keyword()}]) ::
  {:ok, map()} | {:error, term()}

Creates multiple groups and/or arrays in parallel.

Reduces latency for cloud storage by writing metadata concurrently. Returns a map of created items keyed by their names.

Examples

items = [
  {:group, "exp1"},
  {:group, "exp2"},
  {:array, "exp1/results", shape: {100, 100}, chunks: {10, 10}, dtype: :float32}
]

{:ok, created} = Group.batch_create(root, items)
# created = %{
#   "exp1" => %Group{...},
#   "exp2" => %Group{...},
#   "exp1/results" => %Array{...}
# }

create(group_path, opts \\ [])

@spec create(
  String.t(),
  keyword()
) :: {:ok, t()} | {:error, term()}

Creates a new group.

Options

  • :storage - Storage backend (default: :memory)
  • :path - Path for filesystem storage

Examples

{:ok, group} = ExZarr.Group.create("/data", storage: :filesystem, path: "/tmp/zarr")

create_array(group, name, opts)

@spec create_array(t(), String.t(), keyword()) ::
  {:ok, ExZarr.Array.t()} | {:error, term()}

Creates a new array within this group.

Examples

{:ok, array} = ExZarr.Group.create_array(group, "measurements",
  shape: {1000},
  chunks: {100},
  dtype: :float64
)

create_group(parent_group, name, opts \\ [])

@spec create_group(t(), String.t(), keyword()) :: {:ok, t()} | {:error, term()}

Creates a subgroup within this group.

Examples

{:ok, subgroup} = ExZarr.Group.create_group(group, "experiments")

fetch(group, key)

Access nested groups/arrays using path notation. Creates intermediate groups on write operations.

Examples

# Read access
array = group["temperature"]
nested = group["experiments/exp1/results"]

# Write access (auto-creates intermediate groups)
group = put_in(group["new/path"], array)

# Update access
group = update_in(group["data"], fn arr -> modify(arr) end)

get_array(group, name)

@spec get_array(t(), String.t()) :: {:ok, ExZarr.Array.t()} | {:error, :not_found}

Gets an array from the group by name.

get_attr(group, key)

@spec get_attr(t(), String.t()) :: {:ok, term()} | {:error, :not_found}

Gets an attribute from the group.

get_group(group, name)

@spec get_group(t(), String.t()) :: {:ok, t()} | {:error, :not_found}

Gets a subgroup by name.

get_item(group, path)

@spec get_item(t(), String.t()) ::
  {:ok, ExZarr.Array.t() | t()} | {:error, :not_found}

Gets an item (array or group) at the specified path with lazy loading.

Paths can be nested using forward slashes: "exp1/run2/data". Items are loaded from storage on first access and cached.

Examples

{:ok, array} = Group.get_item(root, "temperature")
{:ok, nested} = Group.get_item(root, "experiments/exp1/results")
{:error, :not_found} = Group.get_item(root, "nonexistent")

list_arrays(group)

@spec list_arrays(t()) :: [String.t()]

Lists all arrays in the group.

list_groups(group)

@spec list_groups(t()) :: [String.t()]

Lists all subgroups.

open(group_path, opts \\ [])

@spec open(
  String.t(),
  keyword()
) :: {:ok, t()} | {:error, term()}

Opens an existing group from storage.

put_item(group, path, item)

@spec put_item(t(), String.t(), ExZarr.Array.t() | t()) :: t()

Puts an item (array or group) at the specified path.

Creates intermediate groups as needed. Returns the updated root group.

Examples

group = Group.put_item(root, "new/path/array", array)
group = Group.put_item(root, "experiments/exp1", subgroup)

remove_item(group, path)

@spec remove_item(t(), String.t()) :: t()

Removes an item from the group.

Does not delete from storage, only removes from the in-memory structure. The path remains in _loaded to indicate it was checked.

Examples

group = Group.remove_item(root, "temperature")
group = Group.remove_item(root, "experiments/exp1")

require_group(parent, path)

@spec require_group(t(), String.t()) :: {:ok, t()} | {:error, term()}

Ensures a group exists at the specified path, creating it if necessary.

Similar to mkdir -p, this creates all intermediate groups along the path. If the path exists and is a group, returns it. If it's an array, returns an error.

Examples

{:ok, group} = Group.require_group(root, "exp1/run2/results")
{:ok, existing} = Group.require_group(root, "experiments")
{:error, :path_is_array} = Group.require_group(root, "temperature")

set_attr(group, key, value)

@spec set_attr(t(), String.t(), term()) :: t()

Sets an attribute on the group.

tree(group, opts \\ [])

@spec tree(
  t(),
  keyword()
) :: String.t()

Generates an ASCII tree visualization of the group hierarchy.

Uses box-drawing characters to show the structure. Arrays are marked with [A] and groups with [G]. Optionally shows array shapes.

Options

  • :depth - Maximum depth to display (default: unlimited)
  • :show_shapes - Include array shapes in output (default: true)

Examples

IO.puts(Group.tree(root))
# Output:
# /
# ├── [A] temperature (1000, 1000)
# ├── [A] pressure (1000, 1000)
# └── [G] experiments
#     └── [G] exp1
#         └── [A] results (500, 500)

IO.puts(Group.tree(root, depth: 2, show_shapes: false))