API Reference macaw_data v#0.18.0

View Source

Modules

The aggregate behavior (spec/04 §1.2): a statistic, one column's readings reduced to one value (visualize#909).

One call in a monitored process of its own, its cost counted rather than timed: the conformance kit's runner (spec/01 §9), ported from visualize's Visualize.Bounded.

One answer of a source: its rows, as the source gave them, and what it actually served (spec/01 §3).

A derived source: a source computed from other sources by a function, rows in and rows out by default, or read as columns under input: :columns (spec/02 §2.2). Ported from visualize's Visualize.VirtualSource (visualize#643, D-194).

A derived function's input read as columns (spec/02 §2.2): accessors over a windowed input table that read its packed buffers and never make a row.

The context a derived source's function of two arguments is called with (spec/02 §2.3): each input's schema as data and its delivery's served facts. It holds no clock and no variables: a variable is an input like any other.

Where derived sources run (spec/02 §8): a value its owner holds and calls from its own callbacks, as a Macaw.Data.Source.Push is held. One owner, and no second process between the owner and its sources' rows. On a Macaw page the owner is macaw_compose's runtime.

A host's derived-source functions (spec/02 §7), written as Elixir in the host's own module and tested with ExUnit

Joined fields (spec/02 §1.4): a reader of an input reads each per-identity source's answer for its own identity. Its rows carry, beside their own fields, every output field but the identity, named "<id>.<field>", a string, never an atom, nil where the output holds nothing for the row's entity; its schema describes them as the outputs do.

A derived source's answer read as any source's (spec/02 §8.5): a Macaw.Data.Source over an engine's last good output at one path, made by Macaw.Data.Derived.Engine.source/3.

The sandbox a derived source's function runs in (spec/02 §6, visualize D-194).

What a stored derived-source function may call (spec/02 §6.2, visualize D-194): the allowlist visualize#643 decided on 2026-10-15, as explicit {function, arity} lists — never "a module minus a few".

Compiled functions by source (spec/02 §6.3, visualize D-194): a value, held by whatever owns the functions (the engine, a builder), not a process or a table, as the schema cache is (spec/01 §1.12).

A stored function checked and compiled once (spec/02 §6.3, visualize D-194): hash, the SHA-256 of its source; fun, the closure; arity; and warnings, the compiler's, each {line, message}, printed nowhere.

What every ! twin raises (spec/00 §5): its value twin's {:error, reason}, with the reason's term kept and its words for the message.

The one interaction shape every component emits (Macaw spec 02 §8).

The fast Fourier transform of a real or complex sequence (spec/04 §4.11): a radix-2 Cooley–Tukey decimation in time over lists, O(N log N), pure Elixir. The engine of the spectrum transform, ported from visualize's Visualize.Data.FFT.

The library's one fault-to-sentence formatter (spec/00 §5): an {:error, reason}'s term as the words a person reads.

How a value is written for a person (spec/05): a format is data, read once from the text it is written in and applied to any number of values.

How a locale writes numbers (spec/05 §3): a d3-format locale definition, as data.

A field's default format (spec/05 §6): how a measure's values are written with no format given, built by Macaw.Data.Format.for_measure/2 and written by Macaw.Data.Format.measure/2.

A time's format (spec/05 §5): a strftime pattern read once into data, written by Macaw.Data.Format.time/2.

The loop over a set of sources, as a value (spec/01 §6.3): subscribe where a source pushes, poll where it cannot, ask each source with a Macaw.Data.Source.Request whose now is the owner's clock, finish each answer, and hand each reader one finished Macaw.Data.Delivery, pooled at the largest budget and each reduced to its own (spec/01 §6.1).

The optional native kernels (spec/04 §1.7): Zig, compiled through Zigler when the build has a Zig of the version Zigler targets, each reading a column's packed Arrow buffers in place and answering exactly what the step's Elixir path answers. Without them every step runs its Elixir path, with the same results.

macaw_data's own package of operations, the registry, and running them (spec/04).

The statistics of macaw_data's package (spec/04 §5), each an aggregate (Macaw.Data.Aggregate) ported from visualize's Visualize.Data

count: the number of readings (spec/04 §5).

delta: the last reading minus the first (spec/04 §5).

deviation: the square root of the sample variance (spec/04 §5).

distinct: the number of distinct readings, a category by spelling (spec/04 §5).

extent: [min, max] of the readings (spec/04 §5).

max: the greatest reading, numbers or times (spec/04 §5).

mean: the arithmetic mean, a float, the exact sum over the count (spec/04 §5).

median: the middle reading, or the mean of the two middles (spec/04 §5).

min: the least reading, numbers or times (spec/04 §5).

quantile: the p-quantile by linear interpolation between the sorted readings, at i = (n − 1) · p, the last index clamped (spec/04 §5).

slope: the least-squares slope of the points the readings are (spec/04 §5): a number v at place i is the point [i, v], a reading [x, y] (or {x, y}) the point itself. nil for fewer than two points, or when every x is equal.

sum: the sum of numbers, 0 of none (spec/04 §5): exact, so any order and any grouping of the readings give the same value. Integers add as integers; with a float among them, the exact sum is rounded once to the nearest float.

variance: the sample variance, Σ(v − mean)² / (n − 1) (spec/04 §5).

A caller's cache of steps' per-chunk parts (spec/04 §1.5; macaw_data#9 §6): the example caller of the caching contract, a plain value the caller holds and passes back. The operations hold no cache and no hidden state; the frame or the feed that owns this value knows when a chunk leaves its window.

An operation's option declaration and the check of a value against it (spec/04 §1.3).

The transforms and aggregates of the enabled packages, by name (spec/04 §2), built by Macaw.Data.Operations.registry/1.

Running a transform a chunk at a time (spec/04 §1.4): a per-chunk part and a merge, so a table's chunks are worked one by one, and a caller may keep each chunk's part across frames.

The table operations of macaw_data's package (spec/04 §4), each a transform (Macaw.Data.Transform) ported from visualize's Chart.Transform.Data, and what they share: finding a column by either spelling of its name, reading a chunk's view of one, a reading as seconds, and a per-row selection. Each is a column-native step (Macaw.Data.Operations.Step, spec/04 §1.4); none reads rows.

aggregate (spec/04 §4.10): one row per distinct value of by, in order of first appearance (a list of fields groups by every one, a nil value a group of its own), holding by's columns and, under as, the named aggregate (fn) of the group's readings of field, nils dropped; without by, one row of as alone. A pair field [x, y] reads each row as the point [x, y], x in seconds when it is a time.

bin (spec/04 §4.9): the readings of field, nils dropped, in bins over their extent: n equal-width bins for an integer thresholds, else at the inner thresholds given. A reading belongs to the first bin [x0, x1) holding it, or to the last when it is the extent's maximum. One row per bin: x0, x1, y0 (0), as (the bin's count, or with fn the named aggregate of its readings) and values.

filter (spec/04 §4.1): the rows, in order, whose reading of field passes test against value. present and absent test nil; eq and ne compare a number by its value (1 equals 1.0) and a category one value whichever spelling it has (Macaw.Data.Operations.same?/2), and a time by its instant; the orderings compare the reading and value by the column's type, by the keys sort orders by (Macaw.Data.Source.Order.key/2), the value read as the column reads it at the edge. So lte and gte both hold exactly when eq does. A nil reading, and one of no reading for its type, never passes an ordering nor eq; a value of no reading for the column's type (nil included) is a fault under an ordering, and under eq and ne on a time column, raised before any row is read.

fold (spec/04 §4.7): wide columns to long rows, one per input row per listed column the row has, in row then fields order; the row's other columns carried, the folded ones dropped by spelling; the column's name under the first name of as, its value under the second. A listed column the rows lack, and a nil reading, yield no row.

The bucket ladder m4 and lttb reduce on (spec/04 §4.4, §4.5; macaw_data#9 §5): one absolute ladder of buckets in the space the x scale maps through, so a chunk's partial depends on its rows, the scale's kind and the level alone, never on the data first read or on the extent.

A ladder piece as one binary (spec/04 §4.5.1; macaw_data#38, #46): its segments as fixed-width records, so a partial the caller's cache holds is a few words on its heap, an off-heap binary beside it, and the merge reads buffers rather than terms. The Elixir walk and the native kernel (Ladder.partial/4) answer the same bytes.

lttb (spec/04 §4.4): shape-preserving downsampling to n rows by MinMaxLTTB (Van Der Donckt et al., 2023), whole rows kept, gaps passing through. n is frame-bound: "plot" is the plot's width in whole pixels. A reducer (spec/04 §7).

m4 (spec/04 §4.5): per pixel column the first, last, least and greatest rows, deduplicated in order, so every column's extremes survive. width is frame-bound. A reducer (spec/04 §7).

sort (spec/04 §4.6): the rows stably sorted by the reading of field under the column's type, ascending or descending, nil last either way: a time by its instant, a number by its value, text and a category by spelling, any other type by term order. The comparison is Macaw.Data.Source.Order's, so the step and a request's row order agree.

spectrum (spec/04 §4.11): a time series to its one-sided amplitude spectrum by FFT (Macaw.Data.FFT.spectrum/3). The interval is every, else the median positive gap of the time column, else 1; the samples are the newest power of two rows with a numeric reading of field; each row holds as's names for the frequency and the magnitude, and a third, when as gives one, for the phase.

stack (spec/04 §4.8): d3-shape's stack (Macaw.Data.Stack) over the series fields, in the order and with the offset given, a nil reading 0. One row per series in fields order, per datum in data order: the datum's columns plus key, series (its stacking position, 0 the bottom), value, y0 and y1.

sum (spec/04 §4.10): aggregate with fn: "sum", as "sum" by default. It is vocabulary a design writes, not a deprecated alias.

take (spec/04 §4.2): the last n rows, or the first with side: :start; n at or above the count is every row and zero is none.

window (spec/04 §4.3): the rows whose reading of field lies in [now − from, now − to), in order. A given to is exclusive, an absent to closes the window at now inclusively, an absent from is open, so adjacent windows partition a column. now is the context's, else the newest reading. With latest, each declared identity's newest row of them (Macaw.Data.Window.latest_rows/3).

What a derived source's stored function calls to reach the registered operations (spec/04 §6): an aggregate of values, or a transform over rows, by name, with the options a design's node would carry.

A parameter: the one shape for every typed option, a source's or a component's (spec/01 §2.2).

A fixed list of quantities, each with its facts: a base unit, a kind, bounds, additivity, and display families by unit system (spec/03 §8–§13).

Shape-preserving downsampling (spec/01 §2.4): the two reducers finishing uses to bring a delivery within a budget.

A source's schema: what each of its fields means (spec/01 §1.2).

Schemas by {feed, version}, as a value (spec/01 §1.12, visualize D-136).

The source contract (spec/01 §2): anything a component's slot reads. One contract serves feeds, queries, derived sources, state sources and exports. It is a strict superset of visualize's Visualize.Feed: an existing feed is a source with no parameters.

One source over several, routed by window (spec/01 §10.2): live data and its history as one source, say an in-process telemetry source holding the last hour in front of a store holding everything.

The source conformance kit: the contract of spec/01 §2–§4 as checks every source runs (spec/01 §9).

What a source is (spec/01 §2.1): its schema, its parameters, the capabilities it honors, and whether it answers every viewer the same.

Finishing (spec/01 §2.4): Macaw does what a source did not.

A request's row order (spec/01 §2.3): what a reader looks at first, so a capped or reduced delivery can hold the first rows in it rather than a sorted sample.

One push subscription, as a value (spec/01 §4): the source's changelog, applied to the relation it holds (spec/01 §2.5).

What a reader asks a source for (spec/01 §2.3).

A generated source (spec/01 §10.1): rows that are a pure function of the tick, so a design under construction moves and replays without a host supplying any data.

An in-process :telemetry collector, one push source per metric (spec/01 §10.3).

d3-shape's stack (spec/04 §4.8): a baseline for each datum, so layers of data stack on one another. The engine of the stack transform, ported from visualize's Visualize.Shape.Stack.

Columnar typed data: the currency of deliveries, derived sources and components (spec/01 §1.1).

One chunk of a Macaw.Data.Table (spec/01 §1.1.1): a run of rows of every column, each column packed by itself in the table's forms, Arrow's record batch. A chunk is immutable.

The transform behavior (spec/04 §1.1): one step of a data pipeline, a table in and a table out, declared once so the validator, a drawer's forms and vocabulary, the JSON forms and the sandbox all read the same declaration (visualize#886).

Units named by their UCUM codes (spec/03 §1–§7, D-138, D-155).

Windows, boxes and regions over a source's rows (spec/01 §2.3, §2.4).