ExZarr.Storage.Backend.ETS (ExZarr v1.1.0)
View SourceETS (Erlang Term Storage) storage backend for Zarr arrays.
Stores chunks and metadata in ETS tables, providing fast in-memory storage with multi-process access capabilities. Unlike the Memory backend which uses an Agent, ETS tables are shared memory accessible by multiple processes.
Configuration
Optional configuration:
:table_name- ETS table name (optional, auto-generated if not provided):named_table- Use named table (optional, default:true):table_type- Table type (optional, default::set):access- Access level (optional, default::public):heir- Heir process for table (optional, default::none)
Dependencies
No external dependencies - ETS is built into Erlang/OTP.
Example
# Register the ETS backend
:ok = ExZarr.Storage.Registry.register(ExZarr.Storage.Backend.ETS)
# Create array with ETS storage
{:ok, array} = ExZarr.create(
shape: {1000, 1000},
chunks: {100, 100},
dtype: :float64,
storage: :ets,
table_name: :my_array_data
)
# Write and read data
ExZarr.Array.set_slice(array, data, start: {0, 0}, stop: {100, 100})
{:ok, result} = ExZarr.Array.get_slice(array, start: {0, 0}, stop: {100, 100})Table Types
:set- One value per unique key (default):ordered_set- Sorted set (slower writes, faster ordered access):bag- Multiple values per key allowed:duplicate_bag- Multiple identical values per key allowed
Access Levels
:public- Any process can read/write (default):protected- Any process can read, only owner can write:private- Only owner process can read/write
Named Tables
Named tables can be accessed by name from any process:
# Create with named table
{:ok, array} = ExZarr.create(
storage: :ets,
table_name: :shared_array,
named_table: true,
access: :public
)
# Access from different process
:ets.lookup(:shared_array, {:chunk, {0}})Table Ownership
ETS tables are owned by the creating process by default. When that process
dies, the table is deleted. Use the :heir option to transfer ownership:
{:ok, array} = ExZarr.create(
storage: :ets,
heir: self()
)Performance Characteristics
- Very fast: Direct memory access, no message passing
- Concurrent: Multiple readers, single writer
- Scalable: Efficient for large datasets
- Non-persistent: Data lost on table deletion
Use Cases
- Multi-process data sharing
- High-performance caching
- Concurrent read workloads
- Inter-process communication
- When you need faster access than Agent/GenServer
Comparison with Memory Backend
| Feature | Memory (Agent) | ETS |
|---|---|---|
| Access | Message passing | Direct memory |
| Concurrency | Sequential | Concurrent reads |
| Speed | Fast | Very fast |
| Sharing | Via messages | Direct access |
| Best for | Single process | Multi-process |
Error Handling
ETS errors are returned as {:error, reason} tuples.
Common errors:
:table_not_found- Table doesn't exist:not_found- Key doesn't exist:access_denied- Permission error
Summary
Functions
Clear all data from an ETS table without deleting it.
Delete an ETS table.
Get the current state (all chunks and metadata) from an ETS table.
Check if an ETS table exists.
Get information about an ETS table.
Functions
Clear all data from an ETS table without deleting it.
Example
:ok = ExZarr.Storage.Backend.ETS.clear_table(:my_array_data)
Delete an ETS table.
Useful for cleanup in tests or when you want to explicitly remove a table.
Example
:ok = ExZarr.Storage.Backend.ETS.delete_table(:my_array_data)
Get the current state (all chunks and metadata) from an ETS table.
Useful for debugging and testing.
Example
state = ExZarr.Storage.Backend.ETS.get_state(:my_array_data)
assert map_size(state.chunks) == 5
Check if an ETS table exists.
Example
if ExZarr.Storage.Backend.ETS.table_exists?(:my_array_data) do
IO.puts("Table exists")
end
Get information about an ETS table.
Example
info = ExZarr.Storage.Backend.ETS.table_info(:my_array_data)
IO.inspect(info.size)