API Reference Arrow v#0.1.0

Pure-Elixir implementation of the Apache Arrow columnar format.

Per-type column structs. Each module under Arrow.Array.* represents one concrete Arrow array.

Variable-length opaque-bytes column. Same layout as Arrow.Array.Utf8 but the value bytes are not constrained to be valid UTF-8.

Boolean column. Values are a packed LSB-first bitmap.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Fixed-point decimal column. Values are little-endian two's-complement integers of the type's bit width (4 / 8 / 16 / 32 bytes per slot). precision and scale come from the column's type; we carry them on the array so it's self-describing.

Fixed-point decimal column. Values are little-endian two's-complement integers of the type's bit width (4 / 8 / 16 / 32 bytes per slot). precision and scale come from the column's type; we carry them on the array so it's self-describing.

Fixed-point decimal column. Values are little-endian two's-complement integers of the type's bit width (4 / 8 / 16 / 32 bytes per slot). precision and scale come from the column's type; we carry them on the array so it's self-describing.

Fixed-point decimal column. Values are little-endian two's-complement integers of the type's bit width (4 / 8 / 16 / 32 bytes per slot). precision and scale come from the column's type; we carry them on the array so it's self-describing.

Dictionary-encoded column. The buffer is just the indices (a primitive integer array referencing entries in a dictionary stored separately, identified by dictionary_id).

64-bit elapsed-time column. Values are little-endian signed int64s in the declared unit.

Fixed-width binary column. values is exactly length * byte_width bytes (no offsets).

Fixed-size list column. The single child array values holds exactly length * list_size items; there are no offsets.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Interval column. The physical layout depends on the variant

Interval column. The physical layout depends on the variant

Interval column. The physical layout depends on the variant

Variable-length opaque-bytes column with 64-bit offsets.

Variable-length list with 64-bit offsets. Same shape as Arrow.Array.List but offsets is length + 1 little-endian int64s.

Variable-length UTF-8 column with 64-bit offsets. Same shape as Arrow.Array.Utf8 but offsets is length + 1 little-endian int64s.

Variable-length list column. offsets is length + 1 little-endian int32s giving the [start, end) index of each slot's items in values (the child array).

Map column. Layout mirrors Arrow.Array.List: a validity bitmap and int32 offsets buffer with a child values array. The child is expected to be an Arrow.Array.Struct whose two members are the key and the value.

All-null column. Carries only a length.

Struct column. children is one inner array per struct member, in the same order as the parent field's children. Every child array's length matches the struct's length.

32-bit time-of-day column. Values are little-endian signed int32s in the declared unit (:second or :millisecond).

64-bit time-of-day column. Values are little-endian signed int64s in the declared unit (:microsecond or :nanosecond).

64-bit timestamp column. Values are little-endian signed int64s in the declared unit. timezone: nil means a naive (unzoned) timestamp.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Fixed-width primitive column. values is a little-endian packed buffer of one element per slot, regardless of validity.

Variable-length UTF-8 column. offsets is length + 1 little-endian int32s; values is the concatenation of every slot's UTF-8 bytes.

Encoders and decoders between in-memory values and Arrow's raw buffer layout.

Error returned by the decoders (Arrow.Ipc.Stream.decode/1, Arrow.Ipc.File.decode/1, Arrow.Json.decode/1) and raised by their decode!/1 variants.

A named column within a schema. Holds the logical type, nullability, an optional per-field metadata map, the child fields used by nested types (List, Struct), and an optional Arrow.Type.DictionaryEncoding when the field is dictionary-encoded.

Encodes and decodes the body bytes of an Arrow IPC RecordBatch.

Encodes and decodes the Arrow IPC file format.

Encodes and decodes the Arrow IPC streaming format.

Arrow integration test JSON format.

Parses Arrow integration test JSON into the in-memory data model.

Emits Arrow integration test JSON form from in-memory Arrow.Schema and Arrow.RecordBatch structs.

Logical (null-aware) view of Arrow.Array.* columns.

A batch of columnar data: a schema, a row count, and one column per field.

Ordered list of Arrow.Field plus an optional schema-level metadata map.

Arrow logical types. Each type is a distinct struct under Arrow.Type.*.

Variable-length opaque byte column.

Logical boolean type. Stored as a packed bitmap.

Date column.

Fixed-point decimal column. Stored as a little-endian two's-complement integer of bit_width bits, scaled by 10^-scale.

Dictionary-encoding annotation on a Arrow.Field.

64-bit elapsed-time column in a fixed unit.

Fixed-width opaque-bytes column. Every slot is byte_width bytes.

Fixed-size list of an inner type. Every slot is exactly list_size items; there is no offsets buffer. The inner type is the single child field of the enclosing Arrow.Field.

IEEE-754 floating point. Tier 1: :single, :double.

Signed or unsigned integer of a given bit width.

Interval column. Three concrete physical layouts

Variable-length opaque byte column with 64-bit offsets.

Variable-length list with 64-bit offsets. Otherwise identical to Arrow.Type.List.

Variable-length UTF-8 string column with 64-bit offsets.

Variable-length list of an inner type. The inner type is described by the single child field of the enclosing Arrow.Field.

Map column. Structurally a List<Struct<key, value>> with a single child field named "entries"; the entries struct's two children are the key and value fields.

Logical Null type. Carries no data — only a length.

Tuple-of-columns. Child fields are the struct's members, in declaration order, and live on the enclosing Arrow.Field.

Time-of-day column. The Arrow FlatBuffers table parameterises both the bit width and the unit

64-bit timestamp at a fixed time unit, optionally annotated with a timezone.

Variable-length UTF-8 string column.

Converts an Arrow IPC file to the streaming format or vice versa, preserving schema, record batches, and dictionaries.

Prints a summary of an Arrow IPC file or stream: format, schema (field names, types, nullability), per-batch row counts, and dictionary sizes.