Macaw.Data.Derived.Columns (macaw_data v0.18.0)

View Source

A derived function's input read as columns (spec/02 §2.2): accessors over a windowed input table that read its packed buffers and never make a row.

A function whose source reads its inputs as columns (input: :columns) is handed each input as a Macaw.Data.Table, and reads it through these. A column is named by either spelling, its atom or its string (Macaw.Layers.Names.same?/2), as a row's field is read.

Every accessor answers {:ok, value} or {:error, reason}, and raises on no input. The reasons (format_error/1 words each):

  • {:unknown_column, name}: the table has no column name names;
  • {:wrong_type, name, expected, actual}: a typed accessor read column name, of type actual, which is none of the types in expected;
  • {:type, :name}: a column was named by something other than an atom or a string;
  • {:type, :table}: the input given is not a Macaw.Data.Table;
  • {:type, :integer}, {:type, :non_neg_integer}: take/2's count is not an integer, or slice/3's start or count is not a non-negative integer.

Each has a ! variant (sum!/2, …) that answers the value alone, or raises Macaw.Data.Error worded by format_error/1.

  • What it holds: names/1, type/2, size/1.

  • Positional reads, one value a row, in order, nil at a gap (false in valid/2), so two columns zip row by row: column/2 the decoded values, as Macaw.Data.Table.column/2 gives them; numbers/2 a number column's; instants/2 a time column's instants, integer Unix microseconds on Macaw.Data.Source.Order.instant/1's scale (a naive time by its wall clock, a date by its midnight), read straight from the buffer; valid/2 the presence mask; and time_kind/2 the kind of time a time column holds, read from its storage form, for host code alone (not on the sandbox allowlist).

  • Reductions over the column's readings, its values present, with the semantics of the package's aggregates (spec/04 §5): count/2, sum/2 (exact, 0 of none), mean/2 (nil of none), and min/2 and max/2 of a number or a time column, the first on a tie, a time compared by its instant and answered as its value. With the native kernels loaded (Macaw.Data.Native), sum, count and mean read an int64 or float64 buffer in the exact-sum kernel, and min and max find their slot in the extremes kernel, which reads an int64, float64 or int32 buffer; their Elixir path reads the same buffers and answers the same.

  • Part of the input: slice/3 and take/2 answer a table of some of its rows, views of its chunks, decoding nothing; rows/1 is the row view, for a function that wants rows after all.

    iex> alias Macaw.Data.Derived.Columns
    iex> table = Macaw.Data.Table.new!([{"v", :number, [1.5, nil, 2.5]},
    ...>   {"t", :time, [~U[2026-01-01 00:00:00Z], ~U[2026-01-01 00:00:01Z], nil]}])
    iex> Columns.sum(table, :v)
    {:ok, 4.0}
    iex> Columns.valid(table, "v")
    {:ok, [true, false, true]}
    iex> Columns.instants!(table, "t")
    [1767225600000000, 1767225601000000, nil]
    iex> Columns.max(table, "t")
    {:ok, ~U[2026-01-01 00:00:01Z]}
    iex> Columns.take!(table, -1) |> Columns.column!("v")
    [2.5]
    iex> Columns.sum(table, "w")
    {:error, {:unknown_column, "w"}}
    iex> Columns.mean(table, "t")
    {:error, {:wrong_type, "t", [:number], :time}}

Summary

Types

A column's name, by either spelling.

Why an accessor answered no value (format_error/1).

An accessor's answer.

Functions

The column's values, in order, nil where one is missing, as Macaw.Data.Table.column/2 decodes them.

The number of the column's readings: its values present.

One reason as a line a person reads.

A time column's instants, in order, nil where one is missing: integer Unix microseconds (Macaw.Data.Source.Order.instant/1), read from the buffer, no time made.

The greatest reading of a number or a time column, the first on a tie, nil of none: a time by its instant (Macaw.Data.Source.Order.instant/1), answered as its value.

The arithmetic mean of a number column's readings, the exact sum over their count; nil of none.

The least reading of a number or a time column, the first on a tie, nil of none: a time by its instant (Macaw.Data.Source.Order.instant/1), answered as its value.

The column names, in order, as the table spells them.

As names/1, the value, or Macaw.Data.Error worded by format_error/1.

A number column's values, in order, nil where one is missing.

The rows, one map a row keyed by every column name (Macaw.Data.Table.rows/1).

As rows/1, the value, or Macaw.Data.Error worded by format_error/1.

The number of rows.

As size/1, the value, or Macaw.Data.Error worded by format_error/1.

The count rows from start (from 0), as a table of the same columns: views of the chunks they lie in, nothing decoded (Macaw.Data.Table.slice/3).

The exact sum of a number column's readings (spec/04 §5): an integer when every reading is one, else the exact sum rounded once; 0 of none.

The first n rows, or with n negative the last -n, as a table of the same columns, as Enum.take/2 takes.

As take/2, the value, or Macaw.Data.Error worded by format_error/1.

The kind of time a :time column holds (macaw_data#149): DateTime (UTC or zoned, whose instants order alike), NaiveDateTime or Date when every chunk holding a value holds that one kind, else :mixed. It is read from each chunk's storage form alone (Macaw.Data.Table.Chunk.time_kind/4), so no value is decoded. A column with no value present, an empty table's included, packs as DateTime and answers it.

The column's declared type (Macaw.Data.Table.types/0).

The column's presence mask, in order: true where a value is present.

Types

name()

@type name() :: atom() | String.t()

A column's name, by either spelling.

reason()

@type reason() ::
  {:unknown_column, term()}
  | {:wrong_type, Macaw.Data.Table.name(), [Macaw.Data.Table.type(), ...],
     Macaw.Data.Table.type()}
  | {:type, :name | :table | :integer | :non_neg_integer}

Why an accessor answered no value (format_error/1).

result(value)

@type result(value) :: {:ok, value} | {:error, reason()}

An accessor's answer.

Functions

column(table, name)

@spec column(Macaw.Data.Table.t(), name()) :: result([term()])

The column's values, in order, nil where one is missing, as Macaw.Data.Table.column/2 decodes them.

column!(table, name)

@spec column!(Macaw.Data.Table.t(), name()) :: [term()]

As column/2, the value, or Macaw.Data.Error worded by format_error/1.

count(table, name)

@spec count(Macaw.Data.Table.t(), name()) :: result(non_neg_integer())

The number of the column's readings: its values present.

count!(table, name)

@spec count!(Macaw.Data.Table.t(), name()) :: non_neg_integer()

As count/2, the value, or Macaw.Data.Error worded by format_error/1.

format_error(arg)

@spec format_error(reason()) :: String.t()

One reason as a line a person reads.

iex> Macaw.Data.Derived.Columns.format_error({:wrong_type, "t", [:number], :time})
~s|column "t" is :time, not :number|

instants(table, name)

@spec instants(Macaw.Data.Table.t(), name()) :: result([integer() | nil])

A time column's instants, in order, nil where one is missing: integer Unix microseconds (Macaw.Data.Source.Order.instant/1), read from the buffer, no time made.

instants!(table, name)

@spec instants!(Macaw.Data.Table.t(), name()) :: [integer() | nil]

As instants/2, the value, or Macaw.Data.Error worded by format_error/1.

max(table, name)

@spec max(Macaw.Data.Table.t(), name()) ::
  result(number() | DateTime.t() | NaiveDateTime.t() | Date.t() | nil)

The greatest reading of a number or a time column, the first on a tie, nil of none: a time by its instant (Macaw.Data.Source.Order.instant/1), answered as its value.

max!(table, name)

@spec max!(Macaw.Data.Table.t(), name()) ::
  number() | DateTime.t() | NaiveDateTime.t() | Date.t() | nil

As max/2, the value, or Macaw.Data.Error worded by format_error/1.

mean(table, name)

@spec mean(Macaw.Data.Table.t(), name()) :: result(float() | nil)

The arithmetic mean of a number column's readings, the exact sum over their count; nil of none.

mean!(table, name)

@spec mean!(Macaw.Data.Table.t(), name()) :: float() | nil

As mean/2, the value, or Macaw.Data.Error worded by format_error/1.

min(table, name)

@spec min(Macaw.Data.Table.t(), name()) ::
  result(number() | DateTime.t() | NaiveDateTime.t() | Date.t() | nil)

The least reading of a number or a time column, the first on a tie, nil of none: a time by its instant (Macaw.Data.Source.Order.instant/1), answered as its value.

min!(table, name)

@spec min!(Macaw.Data.Table.t(), name()) ::
  number() | DateTime.t() | NaiveDateTime.t() | Date.t() | nil

As min/2, the value, or Macaw.Data.Error worded by format_error/1.

names(table)

The column names, in order, as the table spells them.

names!(table)

@spec names!(Macaw.Data.Table.t()) :: [Macaw.Data.Table.name()]

As names/1, the value, or Macaw.Data.Error worded by format_error/1.

numbers(table, name)

@spec numbers(Macaw.Data.Table.t(), name()) :: result([number() | nil])

A number column's values, in order, nil where one is missing.

numbers!(table, name)

@spec numbers!(Macaw.Data.Table.t(), name()) :: [number() | nil]

As numbers/2, the value, or Macaw.Data.Error worded by format_error/1.

rows(table)

@spec rows(Macaw.Data.Table.t()) :: result([map()])

The rows, one map a row keyed by every column name (Macaw.Data.Table.rows/1).

rows!(table)

@spec rows!(Macaw.Data.Table.t()) :: [map()]

As rows/1, the value, or Macaw.Data.Error worded by format_error/1.

size(table)

The number of rows.

size!(table)

@spec size!(Macaw.Data.Table.t()) :: non_neg_integer()

As size/1, the value, or Macaw.Data.Error worded by format_error/1.

slice(table, start, count)

The count rows from start (from 0), as a table of the same columns: views of the chunks they lie in, nothing decoded (Macaw.Data.Table.slice/3).

slice!(table, start, count)

As slice/3, the value, or Macaw.Data.Error worded by format_error/1.

sum(table, name)

@spec sum(Macaw.Data.Table.t(), name()) :: result(number())

The exact sum of a number column's readings (spec/04 §5): an integer when every reading is one, else the exact sum rounded once; 0 of none.

sum!(table, name)

@spec sum!(Macaw.Data.Table.t(), name()) :: number()

As sum/2, the value, or Macaw.Data.Error worded by format_error/1.

take(table, n)

The first n rows, or with n negative the last -n, as a table of the same columns, as Enum.take/2 takes.

take!(table, n)

As take/2, the value, or Macaw.Data.Error worded by format_error/1.

time_kind(table, name)

@spec time_kind(Macaw.Data.Table.t(), name()) ::
  result(DateTime | NaiveDateTime | Date | :mixed)

The kind of time a :time column holds (macaw_data#149): DateTime (UTC or zoned, whose instants order alike), NaiveDateTime or Date when every chunk holding a value holds that one kind, else :mixed. It is read from each chunk's storage form alone (Macaw.Data.Table.Chunk.time_kind/4), so no value is decoded. A column with no value present, an empty table's included, packs as DateTime and answers it.

It is for host code (visualize's board placement): a stored function cannot call it, since its answer is a module (spec/02 §6.2).

iex> alias Macaw.Data.Derived.Columns
iex> table = Macaw.Data.Table.new!([{"t", :time, [~N[2026-01-01 00:00:00], nil]}])
iex> Columns.time_kind(table, "t")
{:ok, NaiveDateTime}
iex> Columns.time_kind(Macaw.Data.Table.new!([{"t", :time, [~D[2026-01-01], ~U[2026-01-01 00:00:00Z]]}]), :t)
{:ok, :mixed}

time_kind!(table, name)

@spec time_kind!(Macaw.Data.Table.t(), name()) ::
  DateTime | NaiveDateTime | Date | :mixed

As time_kind/2, the kind, or Macaw.Data.Error worded by format_error/1.

type(table, name)

The column's declared type (Macaw.Data.Table.types/0).

type!(table, name)

As type/2, the value, or Macaw.Data.Error worded by format_error/1.

valid(table, name)

@spec valid(Macaw.Data.Table.t(), name()) :: result([boolean()])

The column's presence mask, in order: true where a value is present.

valid!(table, name)

@spec valid!(Macaw.Data.Table.t(), name()) :: [boolean()]

As valid/2, the value, or Macaw.Data.Error worded by format_error/1.