ExZarr.DataType (ExZarr v1.1.0)
View SourceData type conversion between Zarr v2 and v3 formats.
Zarr v2 uses NumPy-style dtype strings (e.g., "<i4", "<f8", "|b1") while Zarr v3 uses simplified type names (e.g., "int32", "float64", "bool").
This module provides bidirectional conversion between:
- Internal atoms (
:int32,:float64, etc.) - v2 NumPy dtype strings (
"<i4","<f8", etc.) - v3 simplified type names (
"int32","float64", etc.)
Data Type Mapping
| Internal | v2 NumPy | v3 Name |
|---|---|---|
:int8 | "<i1" | "int8" |
:int16 | "<i2" | "int16" |
:int32 | "<i4" | "int32" |
:int64 | "<i8" | "int64" |
:uint8 | "<u1" | "uint8" |
:uint16 | "<u2" | "uint16" |
:uint32 | "<u4" | "uint32" |
:uint64 | "<u8" | "uint64" |
:float32 | "<f4" | "float32" |
:float64 | "<f8" | "float64" |
| :bool | "|b1" | "bool" |
| :complex64 | "<c8" | "complex64" |
| :complex128 | "<c16" | "complex128" |
| :datetime64 | "<M8" | "datetime64" |
| :timedelta64 | "<m8" | "timedelta64" |
Byte Order
v2 uses byte order prefixes:
<: little-endian>: big-endian|: not applicable (single byte or unicode)
v3 does not encode byte order in the type name (little-endian is default).
Examples
# Convert internal atom to v3 string
iex> ExZarr.DataType.to_v3(:int32)
"int32"
# Convert v3 string to internal atom
iex> ExZarr.DataType.from_v3("float64")
:float64
# Convert internal atom to v2 NumPy string
iex> ExZarr.DataType.to_v2(:int32)
"<i4"
# Convert v2 NumPy string to internal atom
iex> ExZarr.DataType.from_v2("<f8")
:float64
Summary
Functions
Returns whether a dtype represents a boolean type.
Returns whether a dtype represents a complex number type.
Returns whether a dtype represents a datetime type.
Convert an Elixir DateTime to microseconds since epoch.
Returns whether a dtype represents a floating point type.
Converts a v2 NumPy dtype string to an internal dtype atom.
Converts a v3 data type string to an internal dtype atom.
Returns the size in bytes for a given dtype.
Convert a datetime value (microseconds since epoch) to an Elixir DateTime.
Pack a value into binary format for a given dtype.
Parse an ISO 8601 datetime string into microseconds since epoch.
Returns whether a dtype represents a signed integer type.
Returns all supported internal dtype atoms.
Returns whether a dtype represents a timedelta type.
Converts an internal dtype atom to a v2 NumPy dtype string.
Converts an internal dtype atom to a v3 data type string.
Unpack a binary value into Elixir terms for a given dtype.
Returns whether a dtype represents an unsigned integer type.
Validates that a dtype is supported.
Types
Functions
@spec bool?(dtype_atom()) :: boolean()
Returns whether a dtype represents a boolean type.
Parameters
dtype- Internal dtype atom
Returns
trueif boolean,falseotherwise
Examples
iex> ExZarr.DataType.bool?(:bool)
true
iex> ExZarr.DataType.bool?(:int32)
false
@spec complex?(dtype_atom()) :: boolean()
Returns whether a dtype represents a complex number type.
Parameters
dtype- Internal dtype atom
Returns
trueif complex,falseotherwise
Examples
iex> ExZarr.DataType.complex?(:complex64)
true
iex> ExZarr.DataType.complex?(:complex128)
true
iex> ExZarr.DataType.complex?(:float32)
false
@spec datetime?(dtype_atom()) :: boolean()
Returns whether a dtype represents a datetime type.
Parameters
dtype- Internal dtype atom
Returns
trueif datetime,falseotherwise
Examples
iex> ExZarr.DataType.datetime?(:datetime64)
true
iex> ExZarr.DataType.datetime?(:int64)
false
@spec datetime_to_micros(DateTime.t()) :: integer()
Convert an Elixir DateTime to microseconds since epoch.
Parameters
datetime- DateTime struct
Returns
- Microseconds since Unix epoch
Examples
iex> ExZarr.DataType.datetime_to_micros(~U[2021-01-01 00:00:00.000000Z])
1609459200000000
@spec float?(dtype_atom()) :: boolean()
Returns whether a dtype represents a floating point type.
Parameters
dtype- Internal dtype atom
Returns
trueif floating point,falseotherwise
Examples
iex> ExZarr.DataType.float?(:float64)
true
iex> ExZarr.DataType.float?(:float32)
true
iex> ExZarr.DataType.float?(:int32)
false
@spec from_v2(v2_dtype_string()) :: dtype_atom()
Converts a v2 NumPy dtype string to an internal dtype atom.
Handles multiple byte order prefixes (<, >, |).
Parameters
dtype_string- v2 NumPy-style dtype string
Returns
- Internal dtype atom
Examples
iex> ExZarr.DataType.from_v2("<i4")
:int32
iex> ExZarr.DataType.from_v2("<f8")
:float64
iex> ExZarr.DataType.from_v2("|u1")
:uint8
iex> ExZarr.DataType.from_v2(">i4")
:int32
@spec from_v3(v3_data_type()) :: dtype_atom()
Converts a v3 data type string to an internal dtype atom.
Parameters
data_type_string- v3 data type name
Returns
- Internal dtype atom
Examples
iex> ExZarr.DataType.from_v3("int32")
:int32
iex> ExZarr.DataType.from_v3("float64")
:float64
iex> ExZarr.DataType.from_v3("uint8")
:uint8
@spec itemsize(dtype_atom() | v3_data_type()) :: 1 | 2 | 4 | 8 | 16
Returns the size in bytes for a given dtype.
Parameters
dtype- Internal dtype atom or v3 type string
Returns
- Size in bytes
Examples
iex> ExZarr.DataType.itemsize(:int32)
4
iex> ExZarr.DataType.itemsize(:float64)
8
iex> ExZarr.DataType.itemsize("int16")
2
@spec micros_to_datetime(integer()) :: {:ok, DateTime.t()} | {:error, term()}
Convert a datetime value (microseconds since epoch) to an Elixir DateTime.
Parameters
micros- Microseconds since Unix epoch
Returns
- DateTime struct or error
Examples
iex> ExZarr.DataType.micros_to_datetime(1609459200000000)
{:ok, ~U[2021-01-01 00:00:00.000000Z]}
Pack a value into binary format for a given dtype.
Parameters
value- Value to packdtype- Internal dtype atom
Returns
- Binary data
Examples
iex> ExZarr.DataType.pack(42, :int32)
<<42, 0, 0, 0>>
iex> ExZarr.DataType.pack(true, :bool)
<<1>>
iex> ExZarr.DataType.pack({3.0, 4.0}, :complex64)
<<0, 0, 64, 64, 0, 0, 128, 64>>
@spec parse_datetime(binary()) :: {:error, :incompatible_calendars | :invalid_date | :invalid_format | :invalid_time | :missing_offset} | {:ok, integer()}
Parse an ISO 8601 datetime string into microseconds since epoch.
Parameters
datetime_string- ISO 8601 formatted string
Returns
{:ok, microseconds}or{:error, reason}
Examples
iex> ExZarr.DataType.parse_datetime("2021-01-01T00:00:00Z")
{:ok, 1609459200000000}
@spec signed?(dtype_atom()) :: boolean()
Returns whether a dtype represents a signed integer type.
Parameters
dtype- Internal dtype atom
Returns
trueif signed integer,falseotherwise
Examples
iex> ExZarr.DataType.signed?(:int32)
true
iex> ExZarr.DataType.signed?(:uint32)
false
iex> ExZarr.DataType.signed?(:float64)
false
@spec supported_types() :: [dtype_atom(), ...]
Returns all supported internal dtype atoms.
Returns
- List of dtype atoms
Examples
iex> ExZarr.DataType.supported_types()
[:int8, :int16, :int32, :int64, :uint8, :uint16, :uint32, :uint64, :float32, :float64, :bool, :complex64, :complex128, :datetime64, :timedelta64]
@spec timedelta?(dtype_atom()) :: boolean()
Returns whether a dtype represents a timedelta type.
Parameters
dtype- Internal dtype atom
Returns
trueif timedelta,falseotherwise
Examples
iex> ExZarr.DataType.timedelta?(:timedelta64)
true
iex> ExZarr.DataType.timedelta?(:int64)
false
@spec to_v2(dtype_atom()) :: v2_dtype_string()
Converts an internal dtype atom to a v2 NumPy dtype string.
Returns little-endian format by default ("<" prefix).
Parameters
dtype_atom- Internal data type atom
Returns
- v2 NumPy dtype string
Examples
iex> ExZarr.DataType.to_v2(:int32)
"<i4"
iex> ExZarr.DataType.to_v2(:float64)
"<f8"
iex> ExZarr.DataType.to_v2(:uint8)
"<u1"
@spec to_v3(dtype_atom()) :: v3_data_type()
Converts an internal dtype atom to a v3 data type string.
Parameters
dtype_atom- Internal data type atom
Returns
- v3 data type string
Examples
iex> ExZarr.DataType.to_v3(:int32)
"int32"
iex> ExZarr.DataType.to_v3(:float64)
"float64"
iex> ExZarr.DataType.to_v3(:uint8)
"uint8"
Unpack a binary value into Elixir terms for a given dtype.
Parameters
binary- Binary data to unpackdtype- Internal dtype atom
Returns
- Unpacked value
Examples
iex> ExZarr.DataType.unpack(<<42, 0, 0, 0>>, :int32)
42
iex> ExZarr.DataType.unpack(<<1>>, :bool)
true
iex> ExZarr.DataType.unpack(<<0, 0, 64, 64, 0, 0, 128, 64>>, :complex64)
{3.0, 4.0}
@spec unsigned?(dtype_atom()) :: boolean()
Returns whether a dtype represents an unsigned integer type.
Parameters
dtype- Internal dtype atom
Returns
trueif unsigned integer,falseotherwise
Examples
iex> ExZarr.DataType.unsigned?(:uint32)
true
iex> ExZarr.DataType.unsigned?(:int32)
false
iex> ExZarr.DataType.unsigned?(:float64)
false
@spec validate(dtype_atom() | String.t()) :: :ok | {:error, :unsupported_dtype}
Validates that a dtype is supported.
Parameters
dtype- Dtype atom, v2 string, or v3 string
Returns
:okif valid{:error, reason}if invalid
Examples
iex> ExZarr.DataType.validate(:int32)
:ok
iex> ExZarr.DataType.validate("float64")
:ok
iex> ExZarr.DataType.validate("<i4")
:ok
iex> ExZarr.DataType.validate(:invalid)
{:error, :unsupported_dtype}