Changelog

View Source

v0.13.0 (2026-11-05)

Errors are values past the readers: every reader has a ! twin, an improper list and arithmetic past a float's range answer faults, and Macaw.Data.Reduce, Macaw.Data.FFT and Macaw.Data.Source's dispatch answer values (#184, #169, #181, #185, #193, #192, #191, #195). A breaking change for a direct caller of Macaw.Data.Reduce or Macaw.Data.FFT, and new ! twins.

Upgrading from 0.12.x

  • Breaking: Macaw.Data.Reduce.lttb/3, Macaw.Data.Reduce.m4/3, Macaw.Data.FFT.magnitudes/1 and Macaw.Data.FFT.window/2 answer {:ok, value} or {:error, reason}. Each answered a bare list. Call lttb!/3, m4!/3, magnitudes!/1 or window!/2 for the old answer. lttb/2 and m4/2 change with them. The lttb and m4 steps keep the shape of their answers.

  • New ! twins: Macaw.Data.Table.from!/2, Macaw.Data.Schema.from_map!/1, Macaw.Data.Schema.from_json!/2 and Macaw.Data.Derived.from_map!/2.

  • Calls that worked near a float's range are now refused, with a fault in place of the answer. 8.0e307 is the bound called half the largest float below.

    • Macaw.Data.Reduce.lttb/3: {:values, :overflow} when count × |x|, count × |y| or 4 × x range × y range reaches 8.0e307.
    • Macaw.Data.Reduce.m4/3: {:values, :overflow} for an x at or above 8.0e307, and {:width, width} for a width at or above it.
    • Macaw.Data.FFT.fft/1: {:values, :overflow} when the absolute parts sum to 8.0e307 or more. Macaw.Data.FFT.magnitudes/1 refuses a term whose square passes a float's range.
    • Macaw.Data.FFT.spectrum/3: {:values, :overflow} when the absolute sum of the readings reaches 1.0e152, and {:every, :overflow} for an interval whose frequencies pass a float's range.
    • The lttb and m4 steps: {[i, "fields"], :overflow} for an x at or above 8.0e307, a number at or above 1.0e301 under a time scale or past a float's range under log, and for lttb a line whose triangles or bucket means could pass the range.
  • Macaw.Data.Operations.validate/3 refuses what is no data in a step's :term option. A function, a PID, a reference, a port or an improper list in filter's value is {[i, "value"], {:type, :term}}. validate/3 accepted them and apply/4 raised, so a test that only validated such a node changes.

  • Macaw.Data.Source.Finish.ceiling/2 refuses a host: or page: that is no positive integer, {[key], {:type, :pos_integer}}, every fault at once. host: 10.5 and host: 0 answered before, and host: :nope applied no limit at all. ceiling(5, host: 0) is now an error, so a caller using 0 for "nothing" must say so itself.

  • Macaw.Data.Source's dispatch answers {:error, :not_a_source} for a term that is no source, from describe/2, fetch/3, subscribe/2 and a describer/1 lookup, holding nothing of the term. unsubscribe/2 answers :ok for such a term. A describer/1 lookup of a non-tuple entry answered {:ok, entry} as if it were a schema, which was a bug. A test asserting FunctionClauseError from any of these changes.

  • An :overflow fault reads "is beyond a float's range" where it read "sums beyond a float's range". A test on those words changes.

  • An improper list answers a fault at every reader, where it raised. Macaw.Data.Table.new/2 has {[], {:type, :columns}} and Macaw.Data.Operations.validate/3 {["nodes"], {:type, :nodes}}, both new. Macaw.Data.Derived.from_map/2 still raises on an improper list inside a map form: that is the kernel's to refuse (spec/00 §5).

  • spec/04 §4.4's lttb tolerance is wider for a sparse-spike series: twice the exact algorithm's error plus 0.005, plus 1/n for each peak kept beyond the exact algorithm's. No picked row changes.

  • Cost at 50k rows, before to after. The direct calls pay for the new extent check; the steps skip it.

    CallReductionsms
    lttb step, kernels, n 10001,079,886 → 1,099,226 (+1.8%)within noise
    m4 step, kernels, width 10001,111,055 → 1,112,078within noise
    lttb step, Elixir path, refs n 1000144,929 → 132,427 (-8.6%)5.4 → 4.4
    Macaw.Data.Reduce.lttb/3 direct, n 1000465,853 → 670,227 (+44%)25.9 → 20.5
    Macaw.Data.Reduce.m4/3 direct, width 1000699,005 → 903,355 (+29%)16.7 → 19.3
    Macaw.Data.FFT.fft/1, 32,768 samplesabout 5.41M → 5.55M (+2.6%)within noise, about 110
    Macaw.Data.FFT.spectrum/3, 32,768 samplesabout 6.21M → 6.39M (+3%)within noise, about 145
    Macaw.Data.Table.source/150,018 → 53,143 (+6%)0.63 → about 1.1

Changes

  • Macaw.Data.Source.Finish.ceiling/2 refuses a host: or page: that is no positive integer (#195), {[key], {:type, :pos_integer}}, every fault at once. A term that isn't a number orders above every number, so host: :nope or host: "10" applied no limit and ceiling(nil, host: :nope) answered {:nope, false}. host: 10.5 and host: 0, which used to answer, are now refused.
  • Macaw.Data.Source.describe/2, fetch/3 and subscribe/2 answer {:error, :not_a_source} for a term that is no source (#191), where they raised FunctionClauseError or UndefinedFunctionError. describer/1 answers it for an entry that is no {source, context} pair: such an entry either raised or, for one that wasn't a tuple, answered {:ok, entry} as if it were a schema. unsubscribe/2 answers :ok for such a term, and for a module with subscribe/2 and no unsubscribe/2. A wrong Description or Request, or a pid that isn't one, stays a programming error (spec/00 §5).
  • Macaw.Data.Reduce.lttb/3, m4/3, Macaw.Data.FFT.magnitudes/1 and window/2 answer {:ok, value} or {:error, reason}, a breaking change (#192). Each had a bare list. Their twins lttb!/3, m4!/3, magnitudes!/1 and window!/2 answer it as before. Each module words its reasons in format_error/1. Reduce.lttb/3 and m4/3 also refuse what used to raise FunctionClauseError, as values: {:n, n}, {:width, width}, {:data, :list}, {:data, :pair} and {:accessor, accessor}. No project in the family calls these directly.
  • The engines answer numbers past a float's range as {:values, :overflow} (#192). Reduce.lttb/3 holds the lttb step's bounds and Reduce.m4/3 refuses an x at half the largest float or past it, as the m4 step does. FFT.fft/1 refuses readings whose absolute parts sum to half the largest float or past it. Each can refuse a call that happened to work near those bounds. FFT.spectrum/3 answers an improper list of values, or of options, as a fault.
  • spec/04 §4.4 states what lttb's tolerance holds for (#193). A random property failed about once in 2,000 sparse-spike series at n of 20. The step kept more peaks than Macaw.Data.Reduce.lttb/3, and the error metric charges a kept peak. No picked row changes: the bound adds 1/n for each peak kept beyond the exact algorithm's.
  • Macaw.Data.Table.from!/2, Macaw.Data.Schema.from_map!/1, Macaw.Data.Schema.from_json!/2 and Macaw.Data.Derived.from_map!/2, the twins spec/00 §5 says every reader has (#185). Each raises Macaw.Data.Error with exactly its reader's faults. A test reads the spec's list and asserts every twin is exported.
  • A step's arithmetic past a float's range is :overflow, never ArithmeticError (#181). mean, median, quantile, variance, deviation, slope and delta answer it as sum did, bin's edges at field, window's bounds at from and to, and spectrum at field, every and time. What a float holds is the BEAM's answer, to the bit, except in one case below. A fault of :overflow reads "is beyond a float's range" where it read "sums beyond a float's range".
  • lttb and m4 answer {[i, "fields"], :overflow} where their arithmetic could pass a float's range (#181). An x at or above half the largest float, a number at 1.0e301 under a time scale or past a float's range under log, and for lttb a line whose triangles or bucket means could pass it. A line near those bounds that happened to work is now refused. Inside them, the rows kept are the same.
  • Macaw.Data.FFT.spectrum/3 refuses readings whose absolute sum reaches 1.0e152 (#181), {:error, {:values, :overflow}}, and an interval whose frequencies are past a float's range, {:error, {:every, :overflow}}. Past that sum a term's square could pass the range, and one check over the readings replaces one at every butterfly. So a spectrum whose terms happened to stay small enough, and which worked, is now refused.
  • An improper list answers its position's fault at every reader (#184, #169). One guard judges a list, and a test makes each list of a valid input of each reader improper on every run. Macaw.Data.Table.new/2 has {[], {:type, :columns}} and Macaw.Data.Operations.validate/3 {["nodes"], {:type, :nodes}}.
  • A step's :term option refuses what is no data (#187). A function, a PID, a reference, a port or an improper list in filter's value is {[i, "value"], {:type, :term}} from validate/3 and apply/4, where apply/4 raised ArgumentError from the step's digest. validate/3 accepted them before, so a test that only validated such a node changes.

v0.12.1 (2026-11-04)

A typed parameter can default to no value, and a push holder, a stack and a schema's map form answer as faults what they raised (#172, #166, #178, #177, #182). default: nil on a typed Macaw.Data.Parameter means optional with no value. Macaw.Data.Source.Push.drain/2 answers every delivery Macaw.Data.Delivery.new/2 accepts, Macaw.Data.Source.Push.trim/2 keeps a box window, Macaw.Data.Stack.generate/2 answers a value it can't stack or a sum past a float's range, and Macaw.Data.Schema.from_map/1 answers a struct at its path. Two random CI failures go with them. This is a patch, and some calls behave differently, including a few that worked: read the guide below.

Upgrading from 0.12.0

  • Macaw.Data.Source.Push.new/3 refuses a bad cache:, limit:, lateness: or chunk_span: at the call, including values that were never read before. A bad limit: on a source with a window field, a bad chunk_span: on a source with no window field and a bad lateness: on a source with no complete_until all worked, and now answer {:error, faults}. The owner accepted this in a patch because only nonsense option values are affected. Pass a value of the right type or drop the option.
  • Macaw.Data.Stack.series/2 raises Macaw.Data.Error instead of ArithmeticError, and Macaw.Data.Stack.generate!/2's message changed. A test that asserts either one changes with it.
  • Macaw.Data.Source.Description.chunk_span/2 answers nil for extreme resolutions, so those chunks align by row count.
  • A float window value past 2^53 spans is its own slot.
  • A struct that enumerates as {key, value} pairs is no description in Macaw.Data.Schema.from_map/1. A MapSet of them read as one, and is now a fault.
  • Macaw.Data.Stack.generate/2 costs more on wiggle, for its checked arithmetic. 50k rows x 3 keys went from 5,404,131 to 9,180,325 reductions; the time in ms is within noise.
  • New API: default: nil on a typed parameter, Macaw.Data.Stack.stacked/2 and stacked!/2.

Changes

  • Added: default: nil on a typed Macaw.Data.Parameter is optional with no value (#172, #175, spec/01 §2.2). A number has nothing to stand for "none", and a sentinel collides with a real value. Macaw.Data.Parameter.check/1 accepts it on any type, beside values that don't list it, and Macaw.Data.Parameter.validate/2 answers nil when it is absent and takes nil given as the same thing. Macaw.Data.Parameter.conforms?/2 also takes a parameter: nil conforms to one whose default is nil, and to nothing else but :term. conforms?(type, nil) is unchanged. A required parameter and one with any other default still refuse nil, and listing nil in the values of a type it is no value of is still a fault. bindable is unchanged. No stored document's digest changes: nothing digested holds a parameter, and no described source could declare this before. Macaw.Data.Source.Conformance's :deliveries check asks each such parameter given nil, on the first request, so a source that declares one and can't read nil is a finding.

  • Fixed: Macaw.Data.Source.Push.drain/2 answers every delivery Macaw.Data.Delivery.new/2 accepts, where it raised (#166, #176, spec/01 §4). A snapshot or an increment of tuple rows without names:, or of tuple rows of another width than their names, is {:table, faults} in errors, Macaw.Data.Delivery.named_rows/1's faults, as rows that don't pack already are. A refused snapshot leaves what is held and the position as they were. A refused increment consumes its number and applies nothing, neither its retractions and replacements nor its complete_until and stale. A property over generated deliveries found the same class elsewhere on the path, each now answered:

    • An increment under a schema whose fields the held rows don't pack under raised MatchError. It is {:table, faults}, and the holding keeps its schema.
    • Macaw.Data.Table.new/2 with a source's origin: raised ArithmeticError on a number in a window column whose span is a Duration, or on a time under a number span. It is the column's own type fault.
    • A retraction whose window value its span can't slot raised. It is {:retract, :not_held, row}.
    • A row map key that spells nothing (a tuple, a reference) raised from Macaw.Data.Table.apply/2, and from Macaw.Data.Table.get/2 as a name. Such a key is no column. A list of structs as an increment's rows raised there too; they read by their fields, as a snapshot's do.
    • A Date as complete_until under a lateness of hours or less raised ArgumentError. The bound is that midnight less the lateness.
    • A number resolution, claim, lateness or window value past what a float's arithmetic holds raised ArithmeticError. Macaw.Data.Source.Description.chunk_span/2 derives no span from a resolution under a nanosecond a row (over a time field) or under 1.0e-280 of the field's units (over a number one), or over 1.0e280 of either, so those chunks are aligned by row count; a value past a float's precision over its span is its own slot; a span reaching back past the earliest DateTime starts there.

    Nothing that worked changes: every resolution, value and row a push could hold before reads and lays out the same.

  • Fixed: Macaw.Data.Stack.generate/2 answers a value it can't stack as a fault, where it raised ArithmeticError (#178, #179, spec/04 §4.8). This also removes a random CI failure: the reader property hit this only when a generated row used a stack key, so its verdict depended on the seed. The property now generates rows under the stack's own keys.

    • A value that is no number, from the default value function or a caller's, is {[key, j], {:reads, :number}}, row j under key, every one at once. generate!/2 raises those faults. The default function still reads nil as 0. A caller's function answering nil answers no number.
    • A sum past a float's range raised too, and is :overflow: {[key, j], :overflow} for a point, {[:order], :overflow} for a series' sum an order reads. The stack step raised the same way from Macaw.Data.Operations.apply/4 and answers {[i, "fields"], :overflow}.
    • Added: Macaw.Data.Stack.stacked/2 and stacked!/2, the value form over a list of any data. series/2 keeps its bare list and is now stacked!/2, so it raises Macaw.Data.Error where it raised ArithmeticError.

    Nothing that worked changes. One stack does no arithmetic, so it takes any value and answers it as it is, as it did: one key under the :none offset and an order of :none, :reverse or {:keys, _}. Every sum a float holds is the BEAM's own answer, to the bit, and integers of any size stay exact. generate!/2's message now starts "not a source it stacks".

  • Fixed: Macaw.Data.Source.Push.trim/2 answers a holding whose served window is a box, where it raised CaseClauseError (#177, #180, spec/01 §4). The chunks wholly before from drop and the box stays as it is, since it has no start to narrow.

  • Changed: Macaw.Data.Source.Push.new/3 refuses an option value it can't use, at the call, and new!/3 raises those faults (#177, #180, spec/01 §4). A call with a bad option value that only failed later now answers {:error, faults} at once.

    • limit: must be an integer, {[:limit], {:type, :integer}}. A float, nil, an atom or a string raised at a drain under a schema with no window field: ArithmeticError, or for a float FunctionClauseError once the rows passed it.
    • lateness: must be nil, a number or a Duration, {[:lateness], {:type, :lateness}}. Anything else recorded {:lateness, :incompatible, claim} on every increment after a claim. A Duration holding no integers raised.
    • chunk_span: must be nil, a positive number or a positive Duration of fixed length, {[:chunk_span], {:type, :chunk_span}}. Anything else was {:table, faults} on every delivery under a window field. A Duration holding no integers raised, also from Macaw.Data.Source.Description.new/1 and Macaw.Data.Table.new/2's origin:, which now answer their fault. An integer span past what a float holds raised ArithmeticError over a float window value, here and from origin:, and now lays chunks out as it does over an integer.
    • cache: must be a Macaw.Data.Schema.Cache, {[:cache], {:type, :cache}}. Anything else raised at a drain.

    Some such calls never reached the failure, and are refused all the same. new/3 knows no schema and no rows, so it can't tell: a bad limit: under a source with a window field, where limit is not read; a float limit: no holding reached; a bad chunk_span: under a source with no window field; a bad lateness: under a source that states no complete_until. Pass a value of the type above, or drop the option. Every other working value stays: a limit: of 0 or below still holds nothing, a negative lateness: still tightens the bound, a Duration of any sign or length is still a lateness:, and a value of the wrong kind for the window field (a number span over a :time field) is still the delivery's fault.

  • Fixed: Macaw.Data.Schema.from_map/1 answers a struct as a description, or as a value entry, as a fault at its path, where it raised Protocol.UndefinedError or FunctionClauseError (#182, #183, spec/01 §1.11). It is {[name], {:type, :description}}, or {[name, :values], {:type, :values}}, and Macaw.Data.Derived.from_map/2 answers it under [:schema]. This also removes a random CI failure: the reader property hit this only when a generated pair was a binary name then a struct, so its verdict depended on the seed. The property now generates both on every run. One call that answered changes: a struct that enumerates as {key, value} pairs, a MapSet of them, read as a description or an entry by those pairs, and is now the same fault.

v0.12.0 (2026-11-03)

A source says its answer is stale, an inverted window is refused, a push holder and a time column read through public seams, and errors are values to the end (#148, #122, #123, #149, #139, #130, #161, #162). A delivery's served.stale says a source answered from a store it could not refresh, carried through finishing, pooling, a push and a derived source's output, so a surface can draw stale rows as stale. Macaw.Data.Window.check/2 and Macaw.Data.Source.fetch/3 refuse a window or region that runs backward. Macaw.Data.Source.Push.trim/2, take_errors/1 and pending?/1, Macaw.Data.Derived.Columns.time_kind/2 and Macaw.Data.Table.Chunk.time_kind/4 replace reads of a struct's fields and of internal storage forms. Every ! twin raises Macaw.Data.Error in words, carrying its value twin's reason, and the readers of a caller's rows (Macaw.Data.Table.rows/1, Macaw.Data.Delivery.rows/1, Macaw.Data.Table.column_types/1, time_columns/1, Macaw.Data.Stack.generate/2) answer {:ok, _} or {:error, faults}, with properties over malformed input proving no reader of untrusted input raises. The family's CI scripts are vendored and a drifted copy fails CI and the release. This release is breaking for an adopter that builds a served map by hand, matches a derived function's context exactly, asks a backward window, asserts ArgumentError from a twin, reads rows/1, column_types/1, time_columns/1 or generate/2 as a bare value, or passes Macaw.Data.Table.from/2 an option it ignored. The guide below lists what to change, and the entries after it give the detail.

Upgrading from 0.11.x

  1. A delivery's served map has a stale key (#148, #150). Macaw.Data.Delivery.new/2 fills it, false unless the source gives served: [stale: true], so a source built on it changes nothing. A served map built by hand, around Delivery.new/2, or a match that lists its keys exactly, adds stale. A stale that is not a boolean is the fault {[:served, :stale], {:type, :boolean}}, and Macaw.Data.Source.Conformance's :deliveries check finds it malformed. A reader that draws a delivery reads served.stale to draw a stale answer as stale.
  2. A derived source's output is stale when any input's delivery is, and its function's context has "stale" (#148, #151). An idle input counts, so a derived output over a source serving its last good answer stays stale until that input delivers a fresh one. A change of stale alone marks the readers, as a changed answer. A function of two arguments that compared an input's context map whole, or read its keys as a list, sees the new "stale" key beside "capped"; a map pattern of the keys it reads is unaffected.
  3. An inverted request window is refused (#122, #152). Macaw.Data.Window.check/2 refuses a window whose from is after its to, {[:window], {:window, :inverted}}, as it refused an inverted region range, comparing two times of any kinds on the instant scale. Macaw.Data.Source.Finish.finish/4 and a derived source's request ({[:from, i, :window, :window], {:window, :inverted}}) refuse what they finished to no rows, and Macaw.Data.Source.fetch/3 refuses an inverted window or region range before calling the source, {:error, {:window, faults}}. A caller that asked a backward window and relied on an empty answer orders its ends or matches the fault; an equal pair and an open end still pass.
  4. A ! twin raises Macaw.Data.Error, not ArgumentError (#130, #157). A test that asserts assert_raise ArgumentError on a twin (Macaw.Data.Table.new!/2, Macaw.Data.Delivery.new!/2, Macaw.Data.Source.Request.new!/1 and the others) asserts Macaw.Data.Error. The exception's reason holds the value twin's error term, so a test can match that term rather than a message.
  5. Macaw.Data.Table.rows/1 and Macaw.Data.Delivery.rows/1 answer {:ok, rows} (#130, #158). A caller that reads rows it knows are valid (a table, a delivery Macaw.Data.Delivery.new/2 built) calls the new twin rows!/1: Macaw.Data.Table.rows!(table) is what Macaw.Data.Table.rows(table) was. A caller that reads a source it was given matches {:ok, rows} and {:error, faults}, Macaw.Data.Table.source/1's faults (a delivery's under [:rows]), where it used to catch nothing while the call raised ArgumentError.
  6. Macaw.Data.Table.from/2 refuses an option it does not take (#159). It takes types: alone, and any other key is {:error, [{[key], :unknown_key}]} where it was ignored. Drop the option. types: that is no map is {[:types], {:type, :map}} where it raised BadMapError, here and through Macaw.Data.Delivery.table/2.
  7. Macaw.Data.Table.column_types/1, Macaw.Data.Table.time_columns/1 and Macaw.Data.Stack.generate/2 answer {:ok, value} (#161, #163). A caller whose source is known valid (a table, rows it built) calls the twin, column_types!/1, time_columns!/1 or generate!/2, for what the function returned before; a pipeline stack |> Stack.generate(data) becomes stack |> Stack.generate!(data). A caller reading a source it was given matches {:error, faults}, Macaw.Data.Table.source/1's faults. Stack.generate/2 of a tensor or tuple rows needs Macaw.Data.Table.named/2 first, or a value function (Stack.value/2) that reads them.
  8. Read a push and a time column through their public functions (#123, #149, #153, #154, #155). Nothing breaks, but a holder that read or rewrote %Macaw.Data.Source.Push{} fields calls Macaw.Data.Source.Push.take_errors/1 (which also empties them), pending?/1 and trim/2 instead, and a reader that matched a chunk's internal storage forms for a time column's kind calls Macaw.Data.Derived.Columns.time_kind/2 or Macaw.Data.Table.Chunk.time_kind/4. The CI changes (#160, #164, #165) change nothing a caller sees.

Changes

  • Breaking: Macaw.Data.Table.rows/1 and Macaw.Data.Delivery.rows/1 answer {:ok, rows} or {:error, faults}, each with a rows!/1 twin (#130, #158, spec/01 §1.1, §3, spec/00 §5). A source Macaw.Data.Table.source/1 refuses is answered with its faults where it raised: a column map whose columns differ in length ({[name], {:length, n}}), a column that is no list ({[name], {:type, :column}}), a tensor of rank other than 1 or 2 ({[], {:rank, rank}}) and a struct with no Table.Reader ({[], {:type, :source}}, which used to raise Protocol.UndefinedError). Delivery.rows/1 puts them under [:rows]. Macaw.Data.Delivery.empty?/1 and named_rows/1 read rows the same way. The raise guard's two invariant entries for Table.Edge are gone. Macaw.Data.Table.time_columns/1, Macaw.Data.Table.column_types/1 and Macaw.Data.Stack.generate/2 answer such a source too (#161, below).

  • Breaking: Macaw.Data.Table.column_types/1, Macaw.Data.Table.time_columns/1 and Macaw.Data.Stack.generate/2 answer {:ok, value} or Macaw.Data.Table.source/1's {:error, faults}, each with a ! twin, column_types!/1, time_columns!/1 and generate!/2 (#161, #163, spec/01 §1.1, spec/04 §4.8, spec/00 §5). They read a caller's source through rows!/1 and raised on one source/1 refuses, a column map of unequal columns, a tensor of rank 3, any other term, from functions without !. A table answers {:ok, _} as well, its declared types and time columns. Stack.generate/2 under the default value function, which reads a datum's key by name, answers rows with no names (a tensor's, tuple rows) as {[:names], :required}, where it raised BadMapError; a value function of the caller's still reads them. Macaw.Data.Stack.series/2 is unchanged. A test now lists every call in lib/ of a twin that reads rows (Macaw.Data.Table.rows!/1, Macaw.Data.Delivery.rows!/1, Macaw.Data.Delivery.named_rows!/1) with why its argument can't be refused, since the raise guard sees only literal raise sites. Known: Macaw.Data.Source.Push.drain/2 raises on a snapshot or increment of tuple rows without names (#166).

  • Fixed: no reader of untrusted input raises on input of the right type but the wrong shape (#130, #159, spec/00 §5). Properties over generated malformed input, run over 40 seeds, check Macaw.Data.Table.new/2, Macaw.Data.Table.from/2, Macaw.Data.Schema.new/1, Macaw.Data.Schema.from_map/1, Macaw.Data.Schema.from_json/2, Macaw.Data.Source.Description.new/1, Macaw.Data.Source.Request.new/1, Macaw.Data.Derived.new/1, Macaw.Data.Derived.from_map/2, Macaw.Data.Source.Push.new/3 and Macaw.Data.Source.Finish.ceiling/2. They found these, each now a fault:

  • Breaking: Macaw.Data.Table.from/2 refuses an option it does not take, {[key], :unknown_key}, where it ignored it (#159). It takes types: alone.

  • Known: a map form holding a struct still raises inside macaw_layers' Macaw.Layers.Form.from_map/3, reached through Macaw.Data.Derived.from_map/2. The kernel will refuse it (macaw_layers#37).

  • Breaking: every ! twin raises Macaw.Data.Error in words, with its value twin's {:error, reason} term on the exception's reason (#130, #157, spec/00 §5). Thirteen twins and findings wrote their faults with inspect/1: Macaw.Data.Table.new!/2, Macaw.Data.Source.Request.new!/1, Macaw.Data.Source.Description.new!/1, Macaw.Data.Derived.Engine.source!/3, Macaw.Data.Source.Push.new!/3, Macaw.Data.Source.Push.trim!/2, Macaw.Data.Source.Finish.finish!/4, Macaw.Data.Source.Finish.reduce!/5, Macaw.Data.Source.Finish.ceiling!/2, Macaw.Data.Source.Order.sort!/3, Macaw.Data.Delivery.table!/2, Macaw.Data.Derived.Functions.registry!/1, and the conformance kit's :description_invalid and :probe findings. They now word them through Macaw.Data.Fault.sentence/2, the library's one fault-to-sentence formatter: "not a request: budget: must be a positive integer; now: is required". The other twins keep their module's format_error/1 words. Macaw.Data.Quantity.label!/1 words its reason too. An ArgumentError has no field to carry the term, which is why the exception changed. A stored function's ! call is still the runner's {:error, :raised, module}, now with Macaw.Data.Error as the module.

  • Clarified: a FunctionClauseError from a guard on an argument's type (a map where a list is taken) is a programming error and outside the errors-are-values rule (#130, spec/00 §5). An argument of the right type but the wrong shape is input, and answers a fault.

  • Changed: the raise guard, ci/raises.exs, is the Macaw family's, vendored byte for byte from claude-tools' macaw_ci/raises.exs (#130, #160, spec/00 §5). It also finds exit, the Kernel. and :erlang. forms and Mix.raise, names every raising function in ci/raises_allowlist.exs as %{at: {module, name, arity}, class, reason}, and verifies each class. So each ! twin raises its own Macaw.Data.Error, where five modules' twins shared a private helper. The derived engine's defsource refuses bad code from its macros. The native kernels' build failure is the module's body. The kernels' interface probe sends its answer back instead of exiting with it. A stored function's ! call to Macaw.Data.Ops compiles to the internal __run__!/5, on the allowlist's excluded list beside __run__/5. Nothing a caller sees changes.

  • Changed: the family's other CI scripts are vendored byte for byte from claude-tools' macaw_ci/, so the family's drift check finds none missing or differing (#162, #164, spec/00 §3). CI's new audit job runs ci/advisories.exs over mix hex.audit: a locked package with a security advisory outside ci/advisories_allowlist.exs (empty), or a retired one, fails the pipeline. mix hex.publish first runs ci/release_guard.exs, which refuses a dirty tree, a HEAD that is not the release tag and a tag not on origin. ci/spec_links.exs is the reference copy: it also checks links into another repository's directories and files beyond its spec, repository homes and GitLab issue links, counts each kind, and refuses a link that leaves the repository; CI runs its self-test first. Nothing a caller sees changes.

  • Added: a vendored family script that drifts fails, in CI and before a release (#162, #165, spec/00 §3). The family's drift check ran by hand from claude-tools, which CI can't reach, and nothing in this repository ran it, so the advisory gate and the release guard went missing and the link check fell behind. ci/vendored_manifest.exs holds each vendored copy's SHA-256, and CI's new vendored job (ci/vendored.exs) fails on a copy edited here or missing. mix hex.publish first runs ci/vendored.exs --reference against the family's reference ($MACAW_CI_REFERENCE, else ~/Projects/claude-tools/macaw_ci), which fails on a copy the reference has moved past, a family script not vendored, one the family dropped, and a reference that isn't there. elixir ci/vendored.exs --vendor vendors the reference and rewrites the manifest.

  • Added: served.stale, by which a source says its answer is stale (#148, #150, spec/01 §3). A source that answers from a store it could not refresh (its upstream failed, and it serves its last good answer) gives Macaw.Data.Delivery.new/2 served: [stale: true], beside a note for a person if it likes; it is false by default, and any other value is the fault {[:served, :stale], {:type, :boolean}}, worded "not a boolean". It is a fact of one answer, as capped is, not a capability in supports, so a reader can draw the answer as stale rather than live (macaw's fetch, a follow-up there). Macaw.Data.Source.Finish carries it through every step unchanged, and each reader of a pooled fetch keeps it through Macaw.Data.Source.Finish.reduce/5 (spec/01 §2.4, §6.1). Macaw.Data.Source.Push's holding takes a snapshot's stale, and each applied increment's replaces it (spec/01 §4). Macaw.Data.Source.Conformance's :deliveries check finds a stale that is not a boolean malformed, as it does capped; a down: source answering a stale delivery is :answered_down, as any answer is (spec/01 §9). A served map built by hand, around Delivery.new/2, now needs the stale key.

  • Added: a derived source's output is stale when any input's delivery is (#148, #151, spec/02 §8.7), its rows being drawn from that input's, and a function of two arguments reads each input's "stale" in its context beside "capped" (Macaw.Data.Derived.Context, spec/02 §2.3). A derived input passes its output's on down a chain. A change of stale alone is a changed answer and marks the readers, as a watermark-only change does. Unlike the watermark, an idle input still counts: a source that keeps serving its last good answer delivers nothing new, so it goes idle, and leaving it out would draw its stale rows as live. A stored function that matched its context's input map exactly sees the new key.

  • Changed: an inverted request window is refused, as an inverted region range is (#122, #152, spec/01 §2.3). Macaw.Data.Window.check/2 compares a window's ends once they are of the window field's type, two times of any kinds on the instant scale, and a from after its to is {[:window], {:window, :inverted}}; an equal pair (an instant) and an open end pass. So Macaw.Data.Source.Finish.finish/4 and a derived source's request ({[:from, i, :window, :window], {:window, :inverted}}) refuse what they used to finish to no rows. Macaw.Data.Source.fetch/3 refuses an inverted window or region range before calling the source, {:error, {:window, faults}}. Macaw.Data.Source.Conformance's :errors check asks an inverted number window and an inverted pair of instants of two kinds. A caller that asked a backward window and relied on an empty answer now matches the fault.

  • Added: Macaw.Data.Source.Push.take_errors/1 and Macaw.Data.Source.Push.pending?/1, so a holder reads its subscription without the struct's fields (#123, #153, spec/01 §4). take_errors/1 answers {errors, push}, the errors recorded since the last take in the order recorded, and a push holding none, so a long-lived subscription's errors no longer grow without bound. pending?/1 is whether Macaw.Data.Source.Push.drain/2 has anything buffered to apply: a snapshot or an increment received since the last drain, or increments waiting on a gap.

  • Added: Macaw.Data.Source.Push.trim/2 and trim!/2, so a holder whose window slides drops what is before it without the struct (#123, #154, spec/01 §4, §4.2). It answers {:ok, push} without the held chunks wholly before from in the window field's order, through Macaw.Data.Table.trim/2, so nothing is copied and every chunk kept is the same chunk by id and version; a chunk straddling from is kept whole. The served window starts at from, or the held start where later, its upper end and complete_until unchanged. A from not of the window field's type is {:error, [{[:from], {:type, type}}]}, and it is a no-op with nothing held, no window field, a :point one or a nil from. The push keeps the trimmed start: a later retraction of an unheld row before it, or a replace range wholly before it, is counted in ignored as :trimmed instead of being an error; a late insertion before it is still placed; a snapshot clears it.

  • Added: a time column's kind is public, read from its storage form with nothing decoded (#149, #155, spec/01 §1.1.5, spec/02 §2.2). Macaw.Data.Derived.Columns.time_kind/2 answers {:ok, kind}: DateTime (UTC or zoned), NaiveDateTime or Date when every chunk holding a value holds that kind, else :mixed; a column with no value present is DateTime. Its faults are {:unknown_column, name}, {:wrong_type, name, [:time], actual} and {:type, :table}, and time_kind!/2 raises their words. Macaw.Data.Table.Chunk.time_kind/4 answers a chunk run's kind, and Macaw.Data.Table.Chunk.instants/4, which visualize already read, is documented and public. They replace a match on the internal Chunk.storage/4 forms. time_kind/2 reads a 50,000-row, ten-chunk column in 948 reductions (instants/2 takes 113,949). It is not on the sandbox allowlist: excluded as :atoms, since it answers a module.

  • Clarified: where Macaw.Data.Ops' faults are placed (#139, #156, spec/04 §6). Nothing changes. A transform call runs a one-step pipeline, so its faults are at [0, key] and its name's, or an op option's, at [0, "op"]: 0.10.0's rule for a fault naming an op is the step's, the transform's case. An aggregate call is no step, so its faults name none: [] for an unknown name ({[], {:unknown, :aggregate, name}}), [key] for an option and ["values"] for a reading. From a stored function the paths are the same with the reasons spelled, and an unknown name is refused at its line.

v0.11.0 (2026-11-02)

Every kind of time on one instant scale, and a table gathered and reduced on its columns across chunks (#124, #127, #141, #128, #132, #133, #140). A window bound, a window position, latest, the budget's reducers and min/max/extent read a Date, a NaiveDateTime and a DateTime on one scale, Macaw.Data.Source.Order.instant/1's integer Unix microseconds, so mixing kinds of time answers a value where it raised or was refused. Rows in another order across chunks are gathered by their slots, a one-field order sorts its column's keys, the m4/lttb ladder reads a time x as integers, and a budget is reduced over the table's own columns: on visualize#998's 50,000-row loop the finish of a frame reduced by lttb costs about a quarter of the reductions it did before #146 on the Elixir path and a tenth with the kernels, keeping the same rows. This release is breaking for an adopter that reads Macaw.Data.Window.position/1 of a date or naive time, expects a budget over a Date or NaiveDateTime window field to keep 0.10.0's rows, matches the {:window, :mixed} refusal, or relies on a mixed-kind window or min/max/extent failing. The guide below lists what to change, and the entries after it give the detail.

Upgrading from 0.10.x

  1. Macaw.Data.Window.position/1 answers integer Unix microseconds for every kind of time (#127, #135). A Date was Gregorian days and a NaiveDateTime float Gregorian seconds; a DateTime was and is its Unix microseconds, and a number is itself. A caller that read a date's position as days or a naive time's as seconds divides by 86,400,000,000 or 1,000,000 (and shifts the epoch), or calls Macaw.Data.Source.Order.instant/1 and converts.
  2. A Date or NaiveDateTime series under a budget may keep different rows (#127). "lttb" and "m4" preselect on absolute power-of-two buckets, so the rescaled x moves the bucket edges. A series now keeps the same rows whichever kind of time it is given as, the rows a UTC series keeps, and the rows the lttb and m4 transforms keep over the time column. An expectation or golden written from 0.10.0's reduction of a date or naive series changes; a DateTime or number series is unchanged. :latest now keeps the later of two naive times of one identity however close.
  3. A request window compares times of different kinds by instant, and {:window, :mixed} is gone (#124, #134). A bound of another kind of time than the window field's values trims by instant where Macaw.Data.Window.compare/2 raised FunctionClauseError, through Macaw.Data.Source.Finish.finish/4, the Macaw.Data.Window trims and Macaw.Data.Derived.window/3. Macaw.Data.Window.check/2 no longer refuses a region range whose ends are different kinds of time, and Macaw.Data.Source.Conformance no longer calls such a served window malformed: drop any match on {:window, :mixed}, since only an inverted range is refused. A value with no reading on a bound's scale (a number or a string in a window of instants) lies in no window, as nil does. Macaw.Data.Window.comparable?/2 is new.
  4. min, max and extent over a time column of mixed kinds answer a value (#141, #143). Macaw.Data.Operations.apply/4 raised ArgumentError and Macaw.Data.Operations.aggregate/4 answered the fault {["values"], {:reads, :comparable}}. Both now compare by instant and answer the reading itself, of its own kind (of two at one instant, the first). Code that matched that fault for mixed kinds of time matches the value; a number among times, or a time among numbers, is still that fault.

Changes

  • Changed: a table's budget is reduced over its own columns (#132, #146, spec/01 §2.4). Macaw.Data.Source.Finish.finish/4 and Macaw.Data.Source.Finish.reduce/5 packed each series and number field into a fresh table of positions, values and indexes before the reducer ran. The reducer now reads each series' columns where they lie: x the window field's column, whose time buffer holds the positions, y the field's, beside the rows' indexes. The kept rows are the same. A window field whose column is neither :time nor :number is still read value by value. The ladder's extent over an unsorted time x reads its instants' least and greatest in one pass (Macaw.Data.Operations.Transforms.Ladder). Reductions before and after, on visualize#998's loop (Macaw.Data.Test.LiveLoop: 50,000 UTC rows, two number fields, budget 4,000), Elixir path (kernels):

    • the frame's finish, reduced by lttb: 8,788,073 to 2,272,032 (8,218,433 to 754,860), ~349 to ~153 ms (~298 to ~28 ms);
    • the frame's finish, reduced by m4: 8,255,767 to 1,738,752 (8,116,467 to 674,868);
    • the whole frame, finish and m4 and lttb through the cache, under lttb: 9,212,945 to 2,696,904 (8,356,191 to 891,523);
    • reduce/5 of a 50,000-row table not known sorted: 8,785,631 to 2,853,474 (8,212,293 to 1,339,034);
    • 50 series of 1,000 rows: 5,484,998 to 4,573,772.
  • Fixed: a gathered chunk is named at its view's first slot, so two gathers of one chunk share an id and a version only where they read the same rows (#132, #145, spec/01 §1.1.3). Macaw.Data.Table.Chunk.gather/5 and Macaw.Data.Table.Chunk.reshape/7 named their output {input id, step key}, versioned by {[input version], step key}, while their callers count the kept rows from the view's first row. Macaw.Data.Source.Finish.finish/4's budget, the m4 and lttb outputs and latest gather so. Two views of one chunk cut at different first rows that kept the same counted rows then got one id and one version over different rows. A cache keyed by them, as Macaw.Data.Operations.Cache is, could reuse another view's part: a window's edge chunk is cut a little later every frame. A view from slot 0 keeps its name {input id, step key}; any other's step key is now {step key, {:from, first}}.

  • Pinned: a finished table keeps the held table's chunks (#132, spec/01 §2.4). Macaw.Data.Source.Finish.finish/4's window keeps every chunk inside it by id and version and cuts only the two edge chunks to views, and a delivery no step acts on is the held table. This was already so in 0.10.0: on visualize#998's loop through Macaw.Data.Source.Push, Macaw.Data.Source.Finish and an Macaw.Data.Operations.Cache of m4 and lttb (Macaw.Data.Test.LiveLoop), an unreduced frame reuses 10 of its 14 parts (8 when a new chunk starts). Reduced by m4 under a budget of 4,000 a frame reuses 8 to 10; reduced by lttb, whose picks move with the window, none. The budget itself costs about 8.3 to 8.9 million reductions a frame on 50,000 rows, the cost visualize measured on its board (#146).

  • Changed: the m4 and lttb ladder's Elixir walk (Macaw.Data.Operations.Transforms.Ladder.partial/4 without the native kernels) reads a packed time x column's instants from its buffer, integer microseconds on Macaw.Data.Source.Order.instant/1's scale, instead of building a DateTime per row and calling DateTime.to_unix/2 (#133, #144, spec/04 §4.5.2). Partials are equal term for term to before and to the kernel's. The unsorted x extent reads the same instants. On visualize#998's loop shape (a Macaw.Data.Source.Push holding of 50,000 UTC rows, one 1,000-row increment a frame, Macaw.Data.Source.Finish.finish/4's window, then m4 and lttb through Macaw.Data.Operations.Cache), a frame's reductions on the Elixir path before and after:

    • cached (10 of 14 parts reused): 1,351,771 to 666,731, ~64 to ~40 ms;
    • full (no cache): 5,228,160 to 1,427,844, ~172 to ~47 ms;
    • one sorted 8,192-row UTC chunk's partial at level 0: 411,913 to 100,008.

    With the kernels loaded the frame is unchanged (~8 ms).

  • Changed: rows of a table in another order across its chunks, a full :order among them, are gathered by their slots (#128, #137, #138, spec/01 §2.4). Each column's pieces are joined into one storage where they share a form a join takes (int64, float64, date, UTC and naive buffers, text, binary and dictionary columns), its validity and precision with it, and gathered in the order. Any other column is read and packed again, that column alone. The output is chunked at Macaw.Data.Table.chunk_rows/0, and every result is Macaw.Data.Table.equal?/2 to before. On 50k rows, reductions before and after on the Elixir path (with the kernels in brackets):

    • a random reorder of an int64, a float64 and a UTC column: 4,088,217 to 1,228,251 (448,160), ~154 to ~10 ms;
    • finishing by a full order by cpu: 5,316,721 to 2,856,710 (2,076,736), ~210 to ~46 ms. Its sort is 1,471,715 of that;
    • a random reorder of a text and a category column: 2,064,177 to 1,156,091 (896,060), ~45 to ~43 ms.

    Text and binary pieces join their data, each piece's offsets moved past the data before it, and dictionary pieces join their codes, moved to one dictionary of every piece's strings where the dictionaries differ (#138). Gathering a text column's single slots reads both offsets in one match, which halves a text reorder's work.

  • Changed: finishing a table by a one-field :order sorts the column's keys, so a full order of a 50k-row table costs less than one transpose (1,705,513 reductions; #128, #140, #142, spec/01 §2.4). Macaw.Data.Source.Finish pairs each row index with its key (a time buffer's instants as they lie, any other column's values keyed by the field's type) and sorts them stably as Macaw.Data.Source.Order.sort_by/2 sorts one key, instead of reading each index through closures (Macaw.Data.Source.Order.sort_with/4, which an order of several fields still uses). The order is exactly Macaw.Data.Source.Order.sort/3's. Gathering a buffer's slots without the kernels copies a slot a step, a run at a time, instead of listing the runs first. On 50k rows, reductions before and after on the Elixir path (with the kernels in brackets):

    • finishing by a full order by cpu: 2,856,710 to 1,500,713 (2,076,736 to 1,356,709); its sort 1,471,715 to 754,291;
    • finishing by a full order by time, descending: 3,908,922 to 1,228,659 (3,752,829 to 1,084,694);
    • an order's head under a budget of 1,000: 1,559,266 to 816,397 (1,528,425 to 810,388);
    • a random reorder of an int64, a float64 and a UTC column: 1,228,254 to 588,159 (448,160, unchanged);
    • a random reorder of a text and a category column: 1,156,091 to 940,059 (896,060, unchanged).

    Wall time moved within the noise of a loaded machine (a full order by cpu ~80 to ~110 ms either way; by time ~130 to ~45 ms).

  • Breaking: Macaw.Data.Window.position/1 places every kind of time on Macaw.Data.Source.Order.instant/1's one scale, integer Unix microseconds (#127, #135, spec/01 §2.4). A NaiveDateTime was float Gregorian seconds, which can't hold a microsecond at today's magnitude, and a Date was Gregorian days. A number is still itself and a DateTime still its Unix microseconds. Upgrade: a caller that read a naive position as seconds or a date's as days divides by 1,000,000 or 86,400,000,000, or calls Macaw.Data.Source.Order.instant/1 and converts.

  • Fixed: :latest in Macaw.Data.Source.Finish.finish/4 keeps the later of two naive times of one identity however close, where naive times a few microseconds apart shared a position and the later in order was kept (#127). A column mixing kinds of time is placed on one axis, where latest compared microseconds against days.

  • Changed (behavior): under a budget, a Date or NaiveDateTime window field's series may reduce to different rows than in 0.10.0 (#127). "lttb" and "m4" preselect on absolute power-of-two buckets, so rescaling x moves the bucket edges. A series now keeps the same rows whichever kind of time it is given as, and the rows the lttb and m4 transforms keep over the time column on their time scale. A DateTime series is unchanged. Latest and the budget read a naive or a date column's buffer as they read a UTC one. On a 50k-row table, reductions before and after:

    • latest over a naive column: 2,009,465 to 609,472 (~57 to ~30 ms);
    • latest over a date column: 1,567,973 to 609,472 (~55 to ~31 ms);
    • a budget of 1,000 over a naive column: 7,085,026 to 5,685,114;
    • a budget of 1,000 over a date column: 6,643,478 to 5,685,069;
    • a UTC column's latest (609,472) and budget (5,685,114) unchanged.
  • Fixed: a request window whose bound is another kind of time than the window field's values no longer raises FunctionClauseError from Macaw.Data.Window.compare/2 (#124, #134, spec/01 §2.4). Times compare on one scale whatever their kinds, Macaw.Data.Source.Order.instant/1's, as sort and filter compare them: a NaiveDateTime or Date bound trims DateTime rows, and a list column of mixed kinds, by instant. Macaw.Data.Source.Finish.finish/4, Macaw.Data.Window.trim/3, Macaw.Data.Window.trim_region/2, Macaw.Data.Window.newer/3, Macaw.Data.Window.before/4 and Macaw.Data.Derived.window/3 answer a value for every request Macaw.Data.Window.check/2 passes. A value with no reading on a closed bound's scale (a number or a string in a window of instants) lies in no window, as nil does. Macaw.Data.Window.latest/2, Macaw.Data.Window.earliest/2 and Macaw.Data.Window.intersect/2 keep the first of two bounds that share no scale.

  • Added: Macaw.Data.Window.comparable?/2, whether two values share the scale Macaw.Data.Window.compare/2 compares them on: both numbers, or both instants of any kinds (#124).

  • Fixed: an aggregate step's min, max or extent over a time column of mixed kinds no longer raises ArgumentError from Macaw.Data.Operations.apply/4 (#141, #143, spec/04 §5). A typed :time column passed the step's reading check by its type, and the fold raised on the second kind. Now these statistics compare times of any kinds on one scale, Macaw.Data.Source.Order.instant/1's, as Macaw.Data.Window.compare/2 does (#124). They answer the reading itself, of its own kind, and of two readings at one instant the first. Macaw.Data.Operations.aggregate/4 agrees: it answered the fault {["values"], {:reads, :comparable}} for mixed kinds and now answers the value. A number among times, or a time among numbers, is still that fault. A property now checks that Macaw.Data.Operations.apply/4 never raises over a mixed-kind time column for every aggregate, grouped or not.

  • Changed: Macaw.Data.Window.check/2 no longer refuses a region range whose two ends are different kinds of time ({:window, :mixed}). They compare on the instant scale, so only an inverted range is refused (#124). A served window of two kinds of time is no longer malformed to Macaw.Data.Source.Conformance either.

  • Changed: finishing a table reads a packed time column (UTC, zoned, naive or date) against window bounds of any kind of time as the buffer lies (#124). On a 50k-row UTC table a naive half window costs 319,993 reductions (~8 ms), as a UTC one does (319,867, ~6 ms), and the sorted table's binary search 6,479. Same-kind windows are unchanged (319,860 before).

  • Fixed (docs): two v0.10.0 entries open by saying a wrong-kind filter value raises: the ordering entry (#96, #100) and the eq/ne entry (#102, #109). Since #118 (#114) neither raises. Macaw.Data.Operations.apply/4 answers the fault {[i, "value"], {:reading, type}} once, before any row is read, and Macaw.Data.Operations.apply!/4 raises it, as the v0.10.0 upgrade guide says (#131, #136). The same stale wording is corrected in spec/04, where a step's bad reading is a fault: window ({:reads, :time}), latest without a schema or an identity ({:needs, :schema}, {:needs, :identity}), lttb's y, stack and bin ({:reads, :number}). It is also corrected for Macaw.Data.Derived.Sandbox.compile/2 without a registry ({nil, {:operations, :registry}}) and in Macaw.Data.Operations.Aggregates' docs.

v0.10.0 (2026-11-02)

Errors are values, a derived source reads columns, and a table is finished on its columns (#114, #107, #105). A public function answers bad input with {:error, reason} and keeps its old return in a ! twin; a CI guard holds lib/ to it (spec/00 §5). A derived source may take its inputs as tables (input: :columns) and read them through Macaw.Data.Derived.Columns, which never makes a row. A derived source's output is a table, and the engine windows and finishes a table input on its columns. filter compares by the column's type, and eq/ne compare a time by its instant (#96, #102). This release is breaking for an adopter that calls any function whose return gained {:ok, _}, matches a derived output's rows, writes a derived source document that a 0.9.x reader must read, or relies on filter across kinds. The guide below lists what to change, and the entries after it give the detail.

Upgrading from 0.9.x

  1. A function that answered its value now answers {:ok, value} or {:error, reason} (#114, #117–#121). Where the input is known to be good, call the ! twin, which keeps the 0.9.x return and raises ArgumentError in the module's format_error/1 words. Otherwise match the result. The functions, each with its twin:

    Functions that already answered {:ok, _} or {:error, _} now answer, rather than raise, for the input they used to raise on: Macaw.Data.Operations.apply/4 (a step refusing its input at run time is a fault at its index), Macaw.Data.Table.from/2, the sandbox's check/2, compile/2 and run/3 ({:error, :options, faults}), and Macaw.Data.Delivery.table/2, Macaw.Data.Derived.validate/2 and Macaw.Data.Derived.Engine.new/2 for an option they do not take. A sum beyond a float's range is :overflow where it raised ArithmeticError.

  2. Implementers of the behaviors may answer faults. A transform's run/3 and bind/3 may answer {:error, {key, reason}}, and an aggregate's reduce/2 and finish/2 may answer {:error, :overflow}. Code that calls these callbacks directly matches those answers.

  3. filter across kinds is a fault (#96, #100, #102, #109, #118). An ordering (lt, lte, gt, gte) compares by the column's type, as sort does, and eq/ne on a time column compare by instant. A value that is no value of the column's type (a number against a time, a string against a number, an unparsable time, nil against an ordering) is refused once, before any row is read: apply/4 answers {[i, "value"], {:reading, type}} and apply!/4 raises it. In 0.9.x these answered silently by term order, so a time ordering kept the wrong rows; an expectation written from those rows changes. filter(test: :eq, value: nil) on a time column becomes test: :absent.

  4. A derived source's output is a table (#47, #103, #104). Macaw.Data.Derived.Engine.outputs/1 and Macaw.Data.Derived.Engine.source/2,3's delivery carry a Macaw.Data.Table, so Macaw.Data.Delivery.rows/1 of an output reads as any table's rows: every field present, nil where the function left one out, keys as strings, and a category's atom as its string (%{"state" => "ok"}, not :ok). Match an output's rows on those readings. Change is told by Macaw.Data.Table.equal?/2, so 1 → 1.0 now marks readers, while an atom and its string, or an absent field and nil, do not.

  5. Macaw.Data.Derived.Engine.push/4 tells a changed delivery by value identity (#106, #108). 1.0 re-delivered where 1 was is a change and recomputes; a table and the same rows as a list are two forms, so also a change. One table chunked two ways is not.

  6. Macaw.Data.Derived.window/3 answers a table for a table input under a request window (window, box, region, latest), where it answered the table's rows (#105, #125). Call Macaw.Data.Table.rows/1 on it where you want rows; a list input still answers rows.

  7. Columnar derived sources: the form is version 2, and the kernels' interface is 7 (#107, #110–#113, #115).

    • input: :rows | :columns is new on Macaw.Data.Derived, :rows by default, so an existing source runs as before.

    • to_map/1 and to_json/1 stamp every derived source document version 2. A 0.10 reader reads a version 1 document, but a 0.9.x reader refuses every 0.10 document by name ({:newer_schema, 2, 1}). Upgrade every reader before a writer.
    • The native kernels' interface version is 7. A build with Zig recompiles them with the package; a library built for 0.9.x is refused at start and every step runs its Elixir path (with native: :required, the start fails), so rebuild the dependency (mix deps.compile macaw_data --force) if a stale build lingers.
    • A checker fault naming an allowed module's unknown function whose name is an accessor's names it by atom: {:restricted, {Macaw.Data.Ops, :mean, 1}}. Macaw.Data.Derived.Sandbox.Allowlist.modules/0 lists nine modules.

Changes

  • Changed: Macaw.Data.Source.Finish.finish/4 finishes a table delivery's :box, :region, :order and :budget on its columns too (#105, #126, spec/01 §2.4). No step reads a table's rows, and each keeps exactly the rows it kept before. An order whose head the budget keeps gathers that head alone. On a 50k-row table, reductions before and after:

    • a region keeping half: 3,697,471 to 718,109;
    • an order's top 1,000: 7,851,916 to 1,569,977;
    • a budget of 1,000: 10,263,925 to 5,685,119;
    • a full order: 6,092,786 to 5,316,784.
  • Added: Macaw.Data.Source.Order.sort_with/4 sorts any items by an order under a schema, each field read by a function. Macaw.Data.Source.Order.sort/3 is it over rows (#126).

  • Changed: Macaw.Data.Source.Order.sort_by/2 sorts one key with :lists.keysort/2, stably in either direction, so sort steps and finishing order cost less (#126).

  • Breaking: Macaw.Data.Derived.window/3 answers a table for a table input under a request window (window, box, region, latest), where it answered the table's rows (#105, #125, spec/02 §2.2). A list of rows still answers rows. The derived engine makes an input's rows only at a function's call.

  • Changed: Macaw.Data.Source.Finish.finish/4 finishes a table delivery's :window and :latest on its columns (#105, #125, spec/01 §2.4). It keeps rows of the table's chunks and makes no row, and keeps exactly the rows it kept before. On a 50k-row table, a window keeping half its rows costs 319,799 reductions (5,349,486 before). With the table sorted by its window field, a binary search costs 5,252. Latest costs 609,412 (5,163,695 before).

  • Breaking: a quantity's facts answer {:ok, fact} or {:error, reason} where a map read raised (#114, #121, spec/03 §9–§11). The functions are Macaw.Data.Quantity.base_unit/1, Macaw.Data.Quantity.kind/1, Macaw.Data.Quantity.bounds/1, Macaw.Data.Quantity.additivity/2, Macaw.Data.Quantity.family/2 and Macaw.Data.Quantity.family/3, and the reasons are:

    • {:unknown_quantity, quantity} for a quantity not in all/0;
    • {:unknown_metric_kind, kind};
    • {:unknown_system, system};
    • Macaw.Data.Unit.parse/1's reason for a unit that is no code.

    Each has a ! twin with the old return (Macaw.Data.Quantity.base_unit!/1, Macaw.Data.Quantity.kind!/1, Macaw.Data.Quantity.bounds!/1, Macaw.Data.Quantity.additivity!/2, Macaw.Data.Quantity.family!/2, Macaw.Data.Quantity.family!/3).

  • Breaking: Macaw.Data.Source.Push.new/3 answers {:ok, push}, Macaw.Data.Derived.Engine.source/2,3 {:ok, output} and Macaw.Data.Source.Finish.ceiling/2 {:ok, {budget, clamped?}}, or {:error, faults} (#121, spec/00 §5):

    • an option a function does not take is {[key], :unknown_key};
    • a missing host: is {[:host], :required};
    • a path the engine does not define is {[:path], :unknown_path}.

    Macaw.Data.Source.Push.new!/3, Macaw.Data.Derived.Engine.source!/3 and Macaw.Data.Source.Finish.ceiling!/2 keep the old returns.

  • Changed: Macaw.Data.Delivery.table/2, Macaw.Data.Derived.validate/2 and Macaw.Data.Derived.Engine.new/2 answer an option they do not take as a fault, {[key], :unknown_key} (the engine's {[], [key], :unknown_key}), where Keyword.validate!/2 raised (#121).

  • Changed: the raise guard's allowlist holds only bang helpers, compile-time code and named invariants. The :convert class is refused (#121, spec/00 §5).

  • Breaking: the functions of a unit code answer {:ok, value} or {:error, reason} instead of raising (#114, #120, spec/03 §2, §5, §6):

    Each has a ! twin (Macaw.Data.Unit.dimension!/1, Macaw.Data.Unit.factor!/1, Macaw.Data.Unit.kind!/1, Macaw.Data.Unit.symbol!/2, Macaw.Data.Unit.convert!/3, Macaw.Data.Unit.minor_units!/1) with the old return.

  • Breaking: Macaw.Data.Quantity.label/1, Macaw.Data.Quantity.of_unit/1 and Macaw.Data.Quantity.bounds/2 answer {:ok, value} or {:error, reason} (#120, spec/03 §9, §14). Macaw.Data.Quantity.label!/1, Macaw.Data.Quantity.of_unit!/1 and Macaw.Data.Quantity.bounds!/2 keep the old returns. Macaw.Data.Quantity.bounds/1 is unchanged.

  • Breaking: Macaw.Data.FFT.fft/1 answers {:ok, terms} and Macaw.Data.FFT.spectrum/3 {:ok, lines}, or {:error, reason}: {:samples, n}, {:window, kind}, {:every, every} or {:values, :number}, worded by Macaw.Data.FFT.format_error/1 (#120, spec/04 §4.11). Macaw.Data.FFT.fft!/1 and Macaw.Data.FFT.spectrum!/3 keep the old returns.

  • Breaking: a sum beyond a float's range is the fault :overflow where it raised ArithmeticError (#120, spec/04 §5):

  • Breaking: the sandbox's options are faults, not raises (#120, spec/02 §6.4). Macaw.Data.Derived.Sandbox.check/2 and Macaw.Data.Derived.Sandbox.compile/2 answer {:error, [{nil, {:option, key}}]} and {nil, {:bound, key, ceiling}}, and a compile without a registry {nil, {:operations, :registry}}. Macaw.Data.Derived.Sandbox.run/3 answers {:error, :options, faults}. Macaw.Data.Derived.Engine.new/2 answers a bound above its ceiling as {[], [key], {:bound, key, ceiling}}.

  • Breaking: Macaw.Data.Derived.Functions.registry/1 answers {:ok, registry} or every fault: {[module], :not_a_function_module} and {[name], {:defined_twice, module, other}} (#120, spec/02 §7). Macaw.Data.Derived.Functions.registry!/1 keeps the old return.

  • Changed: the conformance kit's Macaw.Data.Source.Conformance.check/3 answers its own options' faults as findings: {check, {:unknown_check, checks}}, {check, {:option, key}}, {check, {:request, ask}} and {check, {:viewers, :two}}. The test it writes fails with their words, where check/3 raised before (#120, spec/01 §9).

  • Breaking: Macaw.Data.Source.Finish.finish/4 answers {:ok, delivery} or {:error, faults} instead of a delivery or a raise (#114, #119, spec/01 §2.4, spec/04 §7). Every fault is answered before any step runs:

    • a request the schema cannot answer: Macaw.Data.Window.check/2's faults;
    • {[:reduce], {:unknown, :reducer, name}};
    • {[key], :unknown_key} for an option it does not take;
    • tuple rows without names: {[:names], :required};
    • a number field's reading the reducer cannot place: {[:rows, field], {:reads, :number}}.

    Macaw.Data.Source.Finish.reduce/5 answers the same way. Macaw.Data.Source.Finish.finish!/4 and Macaw.Data.Source.Finish.reduce!/5 keep the old returns.

  • Breaking: Macaw.Data.Source.Order.sort/3 answers {:ok, rows}, or {:error, faults} (Macaw.Data.Source.Order.faults/2's: an unknown field, a field of a type with no order, an order of the wrong shape) instead of raising (#119, spec/01 §2.3). Macaw.Data.Source.Order.sort!/3 keeps the old return.

  • Added: Macaw.Data.Table.source/1 checks a source's shape: :ok, or every fault (#119, spec/01 §1.1):

    • {[], {:type, :rows}}: a list mixing row maps and other terms;
    • {[name], {:length, n}}, {[name], {:type, :column}}: a column map's unequal or non-list column;
    • {[], {:rank, rank}}: a tensor of rank other than 1 or 2;
    • {[], {:type, :source}}: any other term.

    Macaw.Data.Table.from/2 answers those faults where it raised. Macaw.Data.Delivery.new/2 answers a column map's or a tensor's shape faults under [:rows], and Macaw.Data.Operations.apply/4 answers them under "source". Macaw.Data.Table.rows/1 reads a source that source/1 accepts.

  • Breaking: Macaw.Data.Operations.apply/4 keeps its own contract: it answers {:ok, table} or {:error, faults} and never raises (#114, #118, spec/04 §3.1). A step that refuses its input at run time stops the run with a fault at its index, where it raised ArgumentError before:

    • {[i, key], {:reads, kind}}: a column holds a reading of the wrong kind (:number, :time, :positive, :comparable, :point);
    • {[i, "value"], {:reading, type}}: filter's cross-kind value (#100, #109);
    • {[i, "latest"], {:needs, :schema | :identity}}: window's latest;

    • {[i, "op"], {:context, :now, :time}} and {[i, "op"], {:unknown, :scale, scale}}: the context;
    • {[i, "op"], :not_a_table}: a transform answering something that is not a table;
    • {["source"], :not_a_table}, or Macaw.Data.Table.from/2's faults under "source": a source that is not a table.

    Macaw.Data.Operations.apply!/4 raises them in Macaw.Data.Operations.format_error/1's words. A typed column is read by its type alone, so a run over typed columns does no extra work: the frame loop's full recompute of its five chains costs 1,936,195 reductions on the Elixir path (1,935,794 to 1,936,190 before) and 179,223 with the kernels (179,160 to 179,193), the same milliseconds within noise.

  • Breaking: Macaw.Data.Operations.registry/1 answers {:ok, registry} or {:error, faults}: {[module], :not_a_package}, {[package, member], :no_behavior}, {[kind, name], {:named_twice, module, other}} and {[:aggregate, name], {:step_key, key}} (#118, spec/04 §2). Macaw.Data.Operations.registry!/1 keeps the old return. Macaw.Data.Operations.registry/0 is unchanged. Macaw.Data.Derived.Engine.new/2 answers a registry fault as {[], [:packages | path], reason}.

  • Breaking: Macaw.Data.Operations.step_digest/2 answers {:ok, digest}, or {:error, {["op"], {:unknown, :transform, op}}} and {:error, {["op"], :required}}. Macaw.Data.Operations.step_digest!/2 keeps the old return (#118).

  • Breaking: Macaw.Data.Operations.Step.run/4 answers {:ok, table} and Macaw.Data.Operations.Step.parts/4 {:ok, parts}, or the step's fault {:error, {key, reason}}. Macaw.Data.Operations.Cache.run/5 answers {:ok, table, cache} or the fault (#118, spec/04 §1.4, §1.5). Macaw.Data.Operations.Step.run!/4, Macaw.Data.Operations.Step.parts!/4 and Macaw.Data.Operations.Cache.run!/5 keep the old returns.

  • Breaking: a transform's run/3 (Macaw.Data.Transform) and bind/3 may answer {:error, {key, reason}}, the step's fault over its input. (#118).

  • Added: an aggregate may declare reads/0, what its readings are read as: :number, :comparable, :point or :any, the default (Macaw.Data.Aggregate.reads/1). Macaw.Data.Operations.aggregate/4 checks the values against it and answers {["values"], {:reads, kind}} (#118, spec/04 §1.2, §5).

  • Breaking: Macaw.Data.Ops.aggregate/2,3 answers {:ok, value} and Macaw.Data.Ops.transform/2,3 {:ok, rows}, or {:error, faults}, instead of raising (#118, spec/04 §6). In a stored function a fault's atoms are spelled as the function's literals are, so the function matches it with its own words. Macaw.Data.Ops.format_error/1 words it. Macaw.Data.Ops.aggregate!/2,3 and Macaw.Data.Ops.transform!/2,3 keep the old returns, and all five are on the sandbox's allowlist. A stored function that wants the bare value moves to the ! twin. Options that name an op are now {[0, "op"], {:invalid_for, name}}.

  • Changed: spectrum's as is checked as one name or a list of two or three ({:list, :name, 2, 3}, a new option type), so a bad list is a validation fault before the step runs (#118, spec/04 §1.3, §4.11).

  • Fixed: grouping by a column the rows lack no longer crashes. Every row's key is nil, as a missing column reads. The property that Macaw.Data.Operations.apply/4 never raises found it (#118).

  • Breaking: Macaw.Data.Delivery.new/2 answers {:ok, delivery} or {:error, faults} instead of a delivery or a raise (#114, #117, spec/01 §3). Each fault is {path, reason}, and all of them are answered at once, in the order of the fields: {[:rows], {:type, :rows}}, {[:rows], :delivery}, {[key], :unknown_key}, {[:served, fact], :unknown_key}, {[:served, fact], {:type, kind}} or {:enum, values}, and the {:type, kind} of names, types, schema, origin, retracted and replace, with an index for a bad retracted row or replacement. Macaw.Data.Delivery.format_error/1 words them. Macaw.Data.Delivery.new!/2 keeps the old return and raises ArgumentError in those words. Upgrade: call new!/2 where the input is known to be good, or match the result.

  • Breaking: Macaw.Data.Delivery.named_rows/1 answers {:ok, rows} or {:error, [{[:names], :required}]} for tuple rows without names, or {:error, [{[:names], {:width, n, width}}]} for rows that don't fit their names, instead of raising (#117). Macaw.Data.Delivery.named_rows!/1 keeps the old return.

  • Measured: Macaw.Data.Delivery.new/2 costs 64 reductions with no options (69 before) and 121 with every option (115 before). Both take 0.6 to 3 µs, the same as before within the noise.

  • Added: errors are values (#114, #116, spec/00 §5). A public function answers bad input with {:error, reason}; a name! twin raises its reason, a macro's misuse is a CompileError, and a raise remains only for a named internal invariant. The CI job raises (ci/raises.exs) fails on a raise, reraise or throw in lib/ outside a bang twin and ci/raises_allowlist.exs. Its 46 :convert entries, the 59 sites still to convert, go as this release's work items land.

  • Added: a :columns source may answer a table or a column map (#107, #113, spec/02 §1.2, §8.2). Macaw.Data.Derived.conform/2 holds the answer to the schema column by column and answers the output table, {:ok, table}; Macaw.Data.Derived.Engine packs it without making a row.

    • A whole column's fault has index nil. The new reasons are :duplicate, :not_a_column and {:length, n}. A table column of another type is {:type, type}.
    • Any other fault is at the lowest index, with an identity gap before a value fault. A single fault is the one the answer's rows would give.
    • A table already in the output's layout is the output as it is.
    • Under per, an entity may answer a table, which is read as its rows.
    • A :rows source still answers rows alone.
  • Faster: answering 5,000 rows of two fields as a column map costs the engine 164,585 reductions to conform and pack, where the same rows cost 976,673.

  • Added: a derived source may read its inputs as columns: input: :rows | :columns on Macaw.Data.Derived (#107, #112, spec/02 §1.1, §2.2, §7, §8.2). It is :rows by default, and a :rows source runs exactly as before.

    • Under :columns, Macaw.Data.Derived.Engine hands the function each input as a Macaw.Data.Table, read with Macaw.Data.Derived.Columns, and makes no row.
    • A table delivery is handed over as windowed. A list delivery, or a request window's finished rows, is packed under the input's schema. Rows that don't pack are the fault :nonconforming, with detail {:input, key, faults} and no value.
    • A stored function's tables are detached (no origin, no chunk identity, no fetch closure), so they hold only their values.
    • Under per, each entity gets a table of its rows, gathered from the input's chunks.
    • The option applies to stored and host functions alike: the source that names a host function says how it is called.
    • new/1 refuses any other value with {[:input], {:enum, [:rows, :columns]}}.
  • Changed: the derived source's map and JSON form is version 2 (#112, spec/02 §1.5). It writes "input": {"$atom": "columns"} for a columnar source and nothing for :rows. A version 1 document reads as a rows source. A reader of version 1 refuses a version 2 document by name ({:newer_schema, 2, 1}).

  • Faster: one recompute of a source summing one field of a 50,000-row, four-column table input, counting the engine's process and the function:

    SourceReductionsms, default heapms, 4M-word heap
    host :rows2,406,074225.381.8
    host :columns, kernels2,3750.90.4
    host :columns, Elixir path152,407n/an/a
    stored :rows11,757,116487.7233.3
    stored :columns, kernels3,6840.50.5
    stored :columns, Elixir path153,716n/an/a

    The :rows rows are unchanged from before. On the Elixir path, only the function's reductions were measured without the kernels.

  • Added: a stored function may read columns: Macaw.Data.Derived.Columns is on the sandbox's allowlist with every accessor, its ! variant and format_error/1 (#107, #111, #115, spec/02 §6.1–§6.3). It is written by its full name, as Macaw.Data.Ops is.

    • The checker holds a literal column name, a string or an atom, to fields: as it holds row.field, so Columns.sum(t, "w") over inputs without w is {:unknown_field, "w"} at its line.
    • Every accessor compiles to a twin in the sandbox's runtime that spells its names, values and reasons by the function's literals' rule. So a stored function's :ok matches a category's "ok", as it does in rows, and {:error, {:unknown_column, _}} matches the reason a missing column answers.
    • A stored function that matches {:ok, v} = on a bad call, or calls a ! variant, raises; the runner answers {:error, :raised, MatchError} or ArgumentError.
    • A map read of a table (t.chunks) raises ArgumentError, so a table's fields are read through the accessors alone.
    • Macaw.Data.Derived.Sandbox.Allowlist.modules/0 now lists nine modules.
  • Changed: a checker fault that names an unknown function of an allowed module whose name is an accessor's (Macaw.Data.Ops.mean(rows)) names it by the atom ({:restricted, {Macaw.Data.Ops, :mean, 1}}), as it names every function the checker knows; it named it by its string. A fault that names a function no allowed module has is unchanged.

  • Added: Macaw.Data.Derived.Columns, a derived function's input read as columns (#107, #110, spec/02 §2.2). It holds accessors over a windowed input table that read its packed buffers and never make a row:

    • names/1, type/2 and size/1;
    • the positional reads column/2, numbers/2, instants/2 (integer Unix microseconds on Macaw.Data.Source.Order.instant/1's scale, read from the buffer with no time made) and valid/2, each nil (or false) at a gap;
    • the reductions count/2, sum/2 (an exact sum, as the sum aggregate's), mean/2, min/2 and max/2, which mean what the aggregates of the same names mean, the first on a tie and a time compared by its instant;
    • slice/3, take/2 and rows/1.

    Every accessor answers {:ok, value} or {:error, reason} and raises on no input (#115). The reasons are {:unknown_column, name}, {:wrong_type, name, expected, actual}, {:type, :name}, {:type, :table}, {:type, :integer} and {:type, :non_neg_integer}. Each accessor has a ! variant that answers the value alone or raises ArgumentError in format_error/1's words. A column is named by either spelling. With the native kernels loaded, sum, count and mean read the exact-sum kernel (group_sum), ungrouped, and min and max read a new kernel, extremes, which answers a view's first least and first greatest slot of an int64, float64 or int32 buffer. The kernels' interface version is now 7. The Elixir path reads the same buffers and answers the same, term for term. The engine's opt-in, input: :columns, comes in #112.

  • Fixed: filter's eq and ne compare a time column by its instant, as its orderings do (#102, #109, spec/04 §4.1). They compared each reading with value by struct equality, so one instant written in another zone or at another precision (~U[2026-01-01 00:00:00Z] and the same instant at +01:00, or at microsecond precision 0 and 6) passed lte and gte yet was not eq, and a naive or date column was never eq a DateTime. Now value is read at the edge as the orderings read it (a DateTime, NaiveDateTime, Date or ISO 8601 string) and compared by Macaw.Data.Source.Order.instant/1, so lte and gte both hold exactly when eq does. ne stays eq's negation: a nil reading passes ne. The rule for every column is stated in spec/04 §4.1: equal on the type's scale, so on a number column 1 equals 1.0 and 0.0 equals -0.0 (unchanged). This is comparison, not identity: 1 → 1.0 is still a change to a derived source's answer and to a pushed delivery (spec/01 §1.1.2, spec/02 §8.2).

  • Breaking (behavior): eq and ne on a time column against a value with no instant (a number, an unparsable string, nil) raise ArgumentError once over the input, before any row is read, with the orderings' message (#102, #109). They used to keep no row (ne: every row). filter(test: :eq, value: nil) on a time column is test: :absent. On any other column a value of another kind is still answered, not refused. Since #118, Macaw.Data.Operations.apply/4 answers this as the fault {[i, "value"], {:reading, type}} and apply!/4 raises it.

  • Faster: eq and ne on a packed time column run in the native kernel's int64 mode on the :utc, :zoned and :naive forms, and without the kernels scan the same buffer (and a date column's days), building no DateTime. Measured over one 8,192-row chunk (apply!/3, the least of 7 runs; the box at load 20, so ms are noisy):

    column, testnative, beforenative, nowElixir path, beforeElixir path, now
    UTC time, eq99,665 r, 2.0 ms1,387 r, 0.08 ms99,666 r, 1.9 ms9,586 r, 0.99 ms
    UTC time, ne117,086 r, 2.5 ms3,618 r, 0.29 ms125,029 r, 2.5 ms25,029 r, 1.1 ms
    UTC time, a tenth missing, eq131,527 r, 3.1 ms1,361 r, 0.05 ms127,922 r, 2.4 ms11,635 r, 0.14 ms
    naive time, eq116,765 r, 2.2 ms1,219 r, 0.05 ms116,706 r, 2.0 ms9,418 r, 1.0 ms
    date, eq82,892 r, 1.6 ms25,594 r, 0.12 ms82,893 r, 1.5 ms25,595 r, 0.12 ms
  • Breaking (behavior): Macaw.Data.Derived.Engine.push/4 tells a changed delivery by value identity, as an answer's change is told (#106, spec/02 §8.2). It compared the whole delivery with ==, which takes 1 for 1.0: a source that re-delivered 1.0 where it delivered 1 marked no reader, and its derived sources kept an answer computed from the old value. Now the rows of two tables compare by Macaw.Data.Table.equal?/2 (so one table chunked two ways is no change, where == of the structs called it one), any other rows by ===, and everything else a delivery carries (served facts, schema, origin, retractions, replacements) by ===. A table and the same rows as a list are two forms, so a change. The answer's watermark is compared exactly too: a complete_until of 1 then 1.0 is a changed answer.

  • Breaking (behavior): a derived source's output is a table, packed once under its output schema, and its change is told by the table's identity (#47, #104, spec/02 §8.2, §8.4, spec/01 §1.1.2). Engine.outputs/1 and Engine.source/2's delivery carry a Macaw.Data.Table, one column a field, named by its string, in name order. So Delivery.rows/1 of an output reads as any table's rows: every field present, nil where the function left one out, and a category's atom as its string (%{"state" => "ok"} where it was :ok). A reader that wants columns no longer packs the rows again: Delivery.table/2 of a 5,000-row output cost 381,292 reductions per read, and the engine now packs it once, for 245,278. A downstream derived source windows it on its columns (#103). The change check is Macaw.Data.Table.equal?/2, not != over the conformed rows: 1 and 1.0 are now a change, which marks the readers. A category's atom and its string, and an absent field and nil, are not a change, where they were. Where you match an output's rows, match the table's reading.

  • Faster: Macaw.Data.Table.equal?/2 of two tables of whole hashed chunks laid out alike compares their chunks' digests and decodes no column. Two 50,000-row canonical tables that differ in one value are told apart in under 1,000 reductions, where comparing their columns decoded both. One recompute of an expression keeping 5,000 of 50,000 input rows: 5,701,833 → 5,944,162 reductions (the packing), 694.6 → 696.4 ms at the default heap.

  • Faster: the derived engine windows a table input as a table (#47, #103, spec/02 §2.2, §8.2). It used to transpose every input with Delivery.rows/1, then sort and group row maps for %{last: n}. Now last: n runs on the columns. It reads the window field as keys (a packed time column's instants straight from its buffer) and the identity fields as values, and skips the sort for a table already sorted by its window field. It keeps the rows as views of the input's chunks, or gathers them by their slots (Chunk.gather/5). A request window still finishes the table's rows, transposed once, as before: finishing a table as a table repacks what it keeps, which costs 37% more (#105 proposes columnar finishing). The rows a function reads are made once per input, at the call, from the windowed table. A row input is handed over as given, as before. One recompute of a last: 20 source over a 50,000-row table input of 50 hosts: 7,923,737 → 915,785 reductions, 322–353 → 16 ms at the default heap, and 146–151 → 9 ms at min_heap_size 4M words. Derived.window/3 takes a table and answers one.

  • Fixed: last: n orders a time by its instant (Source.Order.instant/1), whatever its kind, to the microsecond (#47, #103). It used Window.position/1, which keyed a DateTime in Unix microseconds, a NaiveDateTime in Gregorian seconds as a float and a Date in Gregorian days. So a window field mixing kinds sorted by scale: a Date in 2027 stood before a DateTime in 2026 and was dropped as oldest. Naive times a few microseconds apart also tied, near 6.4e10 seconds, and kept their arrival order. The row window and the columnar one share the key, so they cannot drift. A window field that is neither :time nor :number keeps its rows' order, as a :point field did. So does a value with no reading for its type, such as a number in a :time field, which used to sort as a number.

  • Fixed: filter's orderings compare by the column's type, as sort orders (#96, #100, spec/04 §4.1). lt, lte, gt and gte compared a reading with value by term order, so a time column compared each %DateTime{} as a map, by day of month before year. lt 2026-01-01 kept none of 2026-10-02, 2026-09-30, 2026-10-01, 2025-12-31, and gt kept all four. A naive or date time column against a DateTime kept nothing. The reading and the value are now keyed by Macaw.Data.Source.Order.key/2, the comparator of sort and a request's order, so the three cannot drift: a time by its instant, a number by its value, text and a category by spelling. The value is read as the column reads it at the edge, so against a time column a DateTime, a NaiveDateTime, a Date and an ISO 8601 string are each an instant. A nil reading, and one that is no value of its column's type, never passes.

  • Breaking (behavior): an ordering whose value is no value of the column's type raises ArgumentError once over the input, before any row is read. The message names the step, value, the test, the field and its type. That covers a number against a time column, a string against a number column, an unparsable string against a time column, and nil against any column. Under term order these answered silently: every number stood below every string and atom. A column the rows lack still keeps no row and raises nothing. Where a node compared across kinds on purpose, give it a value of the column's type. Since #118, Macaw.Data.Operations.apply/4 answers this as the fault {[i, "value"], {:reading, type}} and apply!/4 raises it.

  • Changed: Order.key(:time) keys every kind of time on one scale (Order.instant/1, new): Unix microseconds, a NaiveDateTime by its wall clock, a Date at midnight. It was Window.position/1, which keys the three on three scales, so a naive, date or mixed time column sorted wrongly beside DateTimes. A number in a time field is no instant and sorts last.

  • Faster: a time ordering runs in the native kernel on the :utc, :zoned and :naive forms, comparing the column's int64 instants with the value's. No kernel changed: this is the number mode. Without the kernels, the Elixir path scans the same buffer (and a date column's days), gaps or none, and builds no DateTime. Measured over one 8,192-row chunk (apply!/3, gt, the least of 7 runs in reductions and of 21 in ms, on a host at load 25 to 30):

    columnnative, 0.9.0native, nowElixir path, 0.9.0Elixir path, now
    UTC time120,068 r, 3.7 ms3,502 r, 0.17 ms124,987 r, 3.2 ms23,191 r, 0.77 ms
    UTC time, a tenth missing148,710 r, 4.3 ms3,390 r, 0.21 ms177,926 r, 4.6 ms47,898 r, 0.39 ms
    naive time129,004 r, 1.7 ms (wrong rows)3,502 r, 0.17 ms129,008 r, 2.2 ms (wrong rows)23,191 r, 0.77 ms
    date90,802 r, 0.87 ms (wrong rows)33,280 r, 0.26 ms90,806 r, 0.79 ms (wrong rows)38,339 r, 0.24 ms

    Number and text orderings cost what they did: within 3% in reductions, and in ms within the host's noise.

v0.9.0 (2026-11-01)

On macaw_layers 0.4.0, and a request carries a row order (#92, #94). A capped or budgeted delivery now holds the first rows in the order a reader asked for, where it used to hold a sorted sample. The schema's JSON document carries a version stamp and is read through the kernel's gates (#95). A derived source can hold no URL. sort orders a time column by its instant (#93). A raise in a stored function or a conformance callback is answered without logging an [error] (#83). This release is breaking for an adopter that pins macaw_layers ~> 0.3, matches the fault shapes below, or reads Schema.to_json/1's output itself. The guide below lists what to change, and the entries after it give the detail.

Upgrading from 0.8.x

  1. Move to macaw_layers ~> 0.4.0 (#92). macaw_data 0.9.0 requires it, so an application that pins {:macaw_layers, "~> 0.3.0"} (or ~> 0.3) widens the pin to "~> 0.4.0" and follows macaw_layers 0.4.0's own upgrade notes: Form.read/3 takes urls: and refuses every URL without it, and a theme refuses untrusted text. macaw_data builds no theme and its documents hold no URL, so neither change touches what it reads.
  2. Match the new fault shapes.
    • A derived source document's $atom outside the form's atoms is {:unknown_atom, spelling} at its path. It was {:enum, [:identity]}.
    • A URL anywhere in a derived source document ({"$url": …}) is {path, {:url, reason}} at the source's path. A schema document refuses one by type.
    • Schema.from_json/2's malformed JSON is {[], {:json, words}}, where it was {:json, inspect(reason)}. A newer stamp is {[], {:newer_schema, needs, 1}}, a stamp that is no positive integer is {["$version"], {:type, :pos_integer}}, a key beside "$version" and "fields" is {[key], :unknown_key}, and no "fields" is {["fields"], :required}.
    • A request's order that is malformed or that the schema can't answer is {[:order, i], reason} from Request.new/1 and Window.check/2, and {:error, {:order, faults}} from Source.fetch/3.
    • Derived.format_error/1 and Schema.format_error/1 word the kernel's reasons in the kernel's words (Macaw.Layers.Fault). A test that matches the old inspect/1 text needs its expected words updated.
  3. Schema.to_json/1 writes an object (#95): {"$version": 1, "fields": [[name, description], …]}. Where you decode its output yourself, match on the object, not a list. A document written by 0.8.x or earlier still reads: Schema.from_json/2 lifts its list root to version 1. to_map/1 and from_map/1 are unchanged, still the bare pairs, so a derived source's embedded "schema" is unchanged too.
  4. Request.order is new and defaults to [] (#94). A request with no order behaves as before. A source that declares the :order capability must serve its first budget rows in the order and say served.ordered: :source. Otherwise finishing sorts what the source served. Finish.reduce/5 of an ordered delivery keeps the head where it would have reduced by LTTB or M4. Description.capabilities/0 now lists :order, so a test that pins that list needs updating.
  5. sort puts a nil last in either direction (#93). It came first for a time, text or category column ascending, and first for a number column descending. A time column now sorts by its instant, not by struct term order, so an expectation written from the old order changes.

Changes

  • Breaking: the schema's JSON document has a version stamp, read through the kernel's gates (#84, #95, spec/01 §1.11). Schema.to_json/1 writes an object, {"$version": 1, "fields": [[name, description], …]}, so the first change to the form can tell an old document from a new one. to_map/1 and from_map/1 stay the bare pairs, for a document that embeds a schema (a derived source's "schema", unchanged). Schema.from_json/2 reads through Macaw.Layers.Form.gates/3, with no limit or scan code of its own: a newer stamp is refused by name before anything is read, malformed JSON is the kernel's {:json, words} (was {:json, inspect(reason)}), and a key beside "$version" and "fields" is {[key], :unknown_key}. A document written by 0.8.x or earlier still reads: its root is a list, which no version of the object has, so it is lifted to version 1 once its bytes and nesting pass the gates. Match on the object, not a list, where you read to_json/1's output yourself. Schema.format_error/1 words the kernel's reasons where it raised for them.
  • New: a request carries a row order, so a capped delivery holds the first rows in it (#80, #94, spec/01 §2.3–§2.4, §3, §9). Request.order is [{field, :asc | :desc}, …], [] by default. A "slowest first" table under a budget showed a sorted sample, since a source caps before the runtime sees a row (macaw#196). Window.check/2 holds the order's fields to the schema, as {[:order, i], {:unknown_field, name}} or {:invalid_for, type} for :point and :color. Source.fetch/3 refuses such an order before calling the source, {:error, {:order, faults}}, as it refuses parameters. Rows compare by their field's type (Macaw.Data.Source.Order): a time by its instant, a number by value, text by string, a category by its declared order, a missing reading last in either direction, stably.

  • New: the :order capability and served.ordered. A source naming :order serves its first budget rows in the order and says ordered: :source. Otherwise finishing sorts what it served, in a new step before the budget: ordered: :runtime, every row ordered, the head kept; or ordered: :sample, where the source had already cut its rows (reduced_by: :source or capped), so the reader is told they need not be the first. The conformance kit takes order in requests: and checks a source naming :order. A source stating :runtime or :sample is a finding.
  • Changed (behavior): Finish.reduce/5 of an ordered delivery (served.ordered set) keeps its first budget rows, whatever reducer is named, where it reduced by LTTB or M4. A pooled ordered fetch reduces to each reader's own head. Deliveries with no order are reduced as before. Description.capabilities/0 lists eight, :order before :budget.
  • Breaking: on macaw_layers ~> 0.4.0 (was ~> 0.3.0) (#91, #92). An adopter on macaw_layers 0.3.x moves to 0.4.0 with it, following its upgrade notes: Form.read/3 takes urls: and refuses every URL without it, and a theme refuses untrusted text. macaw_data builds no theme and its documents hold no URL, so neither touches what it reads (spec/00 §3).
  • Breaking (faults): a derived source document's $atom outside the form's atoms is {:unknown_atom, spelling} at its path, where it was {:enum, [:identity]} (macawlayers 0.3.1, spec/02 §1.5). Match on `{:unknown_atom, }fromDerived.from_map/2andfrom_json/2`.
  • Fixed: a derived source holds no URL (spec/02 §1.5). macaw_layers 0.4.0 makes $url live in every tag table, and a derived source's reader composes the kernel's gates with new/1 rather than Form.read/3, so a {"$url": …} at a window's bound read into a %Macaw.Layers.Url{} that new/1 accepted, under no host's policy. The reader now holds the source it reads to the kernel's closed policy: a URL, whatever its scheme, is {path, {:url, reason}} at the source's path, and its words never quote it. A schema's reader reads no $url, so one is refused by type (spec/01 §1.11).
  • Changed: Derived.format_error/1 words the kernel's own reasons (an unknown tag or atom, a limit, a newer version, a URL) in the kernel's words (Macaw.Layers.Fault), where it showed them with inspect/1.
  • Fixed: sort orders a column by its type, not by term order (#93, #97, spec/04 §4.6). A time column was sorted by %DateTime{}'s struct term order, so by day of month before year: 2026-10-01, 2026-10-02, 2026-09-30, 2025-12-31 ascending. A time now sorts by its instant, as DateTime.compare/2 orders it, a number by its value, and text or a category by its spelling. :color, :point and :term keep term order. The step shares Macaw.Data.Source.Order's comparison (key/2, sort_by/2, new) with a request's row order, so the two order a field alike. Changed (behavior): a nil reading sorts last in either direction. Before, it came first for a time, text or category column ascending, and first for a number column descending. filter's orderings still compare by term order (#96).
  • Fixed: a raise answered by the bounded runner logs no error (#83, spec/02 §6.4, spec/01 §9). Macaw.Data.Bounded spawned its process with :erlang.spawn_opt/2, so a raise in a stored function, in a compile the compiler refused, or in a conformance callback was also reported by the runtime, as an [error] logged a few milliseconds after the caller had its answer, {:error, :raised, module}. A host running its users' functions logged an error for every user mistake. The runner now spawns through :proc_lib.spawn_opt/2: the answer and the exit reasons are unchanged, and nothing is rescued.
  • Changed (internal): the schema's JSON adapter lives in its own hidden module, Macaw.Data.Schema.Form (#85, spec/01 §1.11). Schema.to_map/1, to_json/1, from_map/1 and from_json/2 stay on Macaw.Data.Schema with their docs, so the public API is unchanged.
  • Tests: the dev-only surfex dependency is ~> 0.11.0 (#90), and WorkTest bounds maps:from_list/1's randomly seeded sort charge (#87). Neither changes the package.
  • A value entry's {"$ramp": [name, t]} reads for any name in a theme's grammar (macaw_layers 0.3.1's Color.Form), as $color does, so a schema's ramp color round-trips through JSON.

v0.8.1 (2026-10-31)

An application without Zig starts with no error logged (#76). Every app on macaw_data without Zigler, the default, logged an [error] crash report at each start, though nothing was wrong. Upgrading from 0.8.0 needs no change; from 0.7.0, follow 0.8.0's guide below.

Changes

  • Fixed: an application without the native kernels starts with nothing logged at error level (#76, #77, spec/04 §1.7). Macaw.Data.Native asked the kernels their interface version at the application's start even when the build had only their stub, and the stub's :not_built raise, in the process that asked, logged a crash report. Macaw.Data.Native.loaded?/0 was false, as it should be. Now only built kernels are asked: a build without them is known at compile time and starts quietly. With Zig, native: :optional and :required behave as before.
  • Fixed: the Hex package never carries Zigler's copy of the kernels' source (#77). Zigler writes .Elixir.<Module>.zig beside a module it compiles, so a package built from a checkout that had compiled the kernels would carry it. The package excludes such files. CI now builds the package, boots a minimal app from it without Zigler, and fails on any [error] line, a loaded?/0 that isn't false, or such a file.
  • Changed: each build stages the native kernels in its own directory (#72, #73, spec/04 §1.7). Zigler stages a module in /tmp/<module> by default, one directory for every checkout, worktree and build path on the host, so two builds at once wrote each other's build.zig and module.zig. The kernels now stage under the build's path, <build path>/zigler/, unless ZIGLER_STAGING_ROOT names another root.
  • Tests: suite verdicts are independent of load and build state (#73, #74, #75, #81): a --warnings-as-errors abort on a warning a lazily loaded test module raised is fixed, LadderNativeTest's property is bounded by its work in reductions rather than ExUnit's clock, and the suite's reduction counter is exact but for maps:from_list/1, whose residue it bounds.

v0.8.0 (2026-10-31)

The source contract is a time-varying relation, and a table is chunked (macaw_data#10, #9), with optional native kernels (#38). A push is a numbered changelog of increments, retractions and range replacements, with settled ranges and derived watermarks. A table is a list of immutable chunks with stable ids, and every step runs a chunk at a time, through a caching contract a caller holds. Where Zig is in the build, the hot steps run natively, bit for bit as their Elixir path, and a 50k-row live frame meets visualize's ~17 ms budget. The suite's verdicts no longer depend on how loaded the host is. On macaw_layers ~> 0.3.0, as 0.7.0 was. This release is breaking: the guide below is what an adopter of 0.7.0 changes, and the entries after it are the detail.

Upgrading from 0.7.0

  1. A table is chunked, opaque, and equal by its contents (#17). Compare tables with Macaw.Data.Table.equal?/2, never ==, since the same rows chunked two ways are one value. Read a table through its functions (rows/1, get/2, column/2, chunks/1), never the struct's fields. Table.t() is opaque. Keep functions, PIDs, references and ports out of tables: new/1 refuses them as {[name, i], :not_data}.
  2. Number a push's increments (#12). A source sends Push.increment_message/3, numbered from 1 per subscription, and may send Push.snapshot_message/2 with served.position. Push.delivery_message/2 is gone. Rows are no longer deduplicated: an identical row pushed again is a second row. Guard redelivery by the number, not by the row.
  3. Push.held/1's rows are a Macaw.Data.Table (#19). Read held rows by name with Macaw.Data.Table.get/2, compare a category as a string, and expect a time column's values as DateTimes. Rows with no window value sort last.
  4. sum and mean are exact (#21). Integers stay an integer. With a float among them, the exact sum is rounded once, ties to even, so a float sum may differ from a sequential sum in the last place: [1.0e16, 1.0, -1.0e16] sums to 1.0, where it was 0.0. A golden written from a sequential float sum may need regenerating.
  5. lttb is MinMaxLTTB, and m4 and lttb reduce on one bucket ladder (#22). lttb keeps at most n rows within its stated tolerance, the same under every chunking. The exact algorithm stays as Macaw.Data.Reduce.lttb/3. m4 is exact over pixel columns whose edges move to a bucket edge, at most a quarter of a pixel. A time column reduces under the :time scale, in microseconds, by default; pass scale: :linear for the former seconds.
  6. The row helpers Transforms.answer/4, point_reader/2 and column/2 are gone (#23). No table operation reads rows any more: a transform reads columns, or declares itself a column-native step (#20).
  7. Don't reuse a cache built by an earlier version. An Operations.Cache's m4/lttb partials changed shape twice (#40, #50), and aggregate, sum, bin and spectrum step keys no longer carry a chunk's place (#24). Drop a held cache on upgrade and let it rebuild.
  8. Native kernels are optional; nothing is needed to keep the Elixir path. To build them, list {:zigler, "~> 0.16.0", runtime: false} in your deps and have Zig 0.16.0 in the build (#48). Zigler's zig_doc pins ex_doc to 0.39.1, so a project that lists ex_doc itself may need override: true on it. Without Zig the build prints a warning, not a compiler diagnostic, and every step runs its Elixir path, with the same results. Set config :macaw_data, native: :required to fail the build, and the start, without the kernels.

Changes

  • Spec: the source contract is one model, a relation that varies over time (#11, #10): a delivery is a snapshot, a push the changelog of Z-set increments, Push.held/1 their integral, :as_of a past snapshot, a derived source a map from relations to relations (spec/01 §2.5). The deferred §2.6 and §4.1–§4.4, and spec/02 §8.6, specify what #12–#15 and macaw_data#9 build: numbered increments and the snapshot's handoff, retractions and range replacements, one resolution per subscription, settled ranges, and derived watermarks.
  • Breaking: push increments are numbered, and a snapshot hands off to the stream by position (#12, spec/01 §4.1). A source sends {:increment, seq, delivery} (Push.increment_message/3, numbered from 1 per subscription) and may send {:snapshot, delivery} (Push.snapshot_message/2) whose served.position (new) is the last increment it reflects; Push.delivery_message/2 is gone. Push drops a repeated or older increment whole (counted in ignored, a count by reason), applies increments in the order of their numbers, holds back those past a gap, records the gap and answers it from Push.refetch/1; a snapshot, received or fetched with the request's new subscription: and handed over by Push.snapshot/2, replaces what is held and drops the increments it already reflects. Rows are no longer deduplicated: an identical row pushed again is a second row (tally taps), so redelivery is guarded by the number, not the row. The conformance kit's :push check holds increments to their numbering. Upgrade: number each subscription's increments from 1 and send them with increment_message/3.
  • New: pushed increments carry corrections (#13, spec/01 §4.2–§4.3). A delivery's retracted rows are withdrawn, each matched by identity, the window value it was held at and the fields it names, one copy per unit of weight, never a guessed row: one not held, held only at another window value, or that can't tell rows apart is {:retract, reason, row} in errors, and against a capped or reduced holding an unheld retraction is counted in ignored (:unheld). Its replace ranges, %{from, to, identities, rows}, make the relation over [from, to), limited to identities, exactly the rows given; a change inside a replaced range, a replacement's row outside it and an overlapping replacement are recorded and not applied. Row retractions apply first, then range retractions, then insertions. An increment is merged only into a relation held at its served.resolution; a mismatch is recorded and answered by Push.refetch/1.
  • New: settled ranges (#14, spec/01 §4.4). A delivery's served.complete_until is the source's watermark, a time or a number below which it expects no further change: settled, not final. Push carries the latest claim forward on what it holds and records a smaller one as {:complete_until, :backwards, held, given}, keeping the held claim; a new subscription starts afresh. A description may declare lateness, a hard bound (a Duration over a time window field, a number over a number one); given to Push.new/3 as lateness:, it refuses a change older than the claim minus the bound ({:late, row, bound}) and a replacement reaching past it whole. Without it a change of any age is accepted.
  • New: derived watermarks (#15, spec/02 §8.6). A derived source's output states served.complete_until, the minimum of its inputs' claims, each capped by its request window's upper bound; an input with no claim leaves none, and a change of the watermark alone marks the readers. A description may declare idle_after (milliseconds), which the owner passes to Engine.new/2 by name (idle_after:): a source that has delivered nothing for that long, judged against the now the engine is given, is left out of the minimum until its next delivery, unless every input is idle.
  • Breaking: a table is chunked, and its identity is its contents (#17, macaw_data#9, spec/01 §1.1.1–§1.1.2; an amendment to dco-tek/macaw/macaw#188 D5). A table is an ordered list of immutable chunks of Macaw.Data.Table.chunk_rows/0 (8,192) rows, each column packed per chunk, so a column's form and a dictionary are a chunk's. append/2, trim/2 and slice/3 keep whole chunks and offsets into them, copying nothing, and chunks/1 lists every chunk a table references. Table.t() is opaque. equal?/2 compares contents strictly and digest/1 is the content digest, the kernel's digest of the canonical term {"macaw.table", [{name, type}, …], [values, …]}. The same rows chunked two ways are one value, and compact/1 rewrites any table into canonical chunks. Each chunk carries its own digest (Table.Chunk.digest/1), computed once when it is built. new/1 refuses a value that is not data, a function, a PID, a reference, a port or an improper list, as {[name, i], :not_data}, since it has no digest. Upgrade: compare tables with Macaw.Data.Table.equal?/2, not ==; read storage through chunks/1, never the struct's fields; keep PIDs, references and functions out of tables.
  • New: chunk identity (#18, macaw_data#9, spec/01 §1.1.3). A chunk has a logical id, where it sits, and a version (Chunk.id/1, Chunk.version/1). A table built from rows names its chunks by their content digests. A source's table, built by Macaw.Data.Table.new/2 with origin: %{source, resolution, window, span}, lays its chunks on fixed slots and names them {source, resolution, slot}: a span's start in event time, else a row offset that is a multiple of chunk_rows/0. So ids hold across a rebuild and a restart, and two resolutions never share one. append/2 keeps a source's alignment, writing rows of the last slot into its chunk again, and compact/1 keeps its slots. A description may declare chunk_span, and Description.chunk_span/2 derives one from the resolution otherwise, never from the data. A step's output chunk (Chunk.derive/3, Chunk.combine/3, gathered by Macaw.Data.Table.from_chunks/3) is named after its inputs and step key. It is versioned by a derivation digest and its contents are not hashed, so a chain of steps hashes nothing.
  • Breaking: Push.held/1's rows are a chunked table (#19, macaw_data#9, spec/01 §1.1.4, §2.6). The rows are a Macaw.Data.Table of the source's origin, its columns the schema's fields, packed as Delivery.table/2 packs them, so a category's atom reads back as its string and a time column's values as DateTimes. Every change reaches the table through the new Macaw.Data.Table.apply/2. It writes an open chunk's new contents and puts a settled chunk's changes (wholly before complete_until) in its delta: a deleted set, predicate deletes for replacements, and a side chunk of insertions, all applied on read. Lazy compaction at a quarter of the base keeps the chunk's id and digest and emits nothing. A cold chunk (Table.Chunk.cold/5) is never fetched for a predicate delete; its digest stays pending until Macaw.Data.Table.warm/1. Push.new/3 takes chunk_span:. Rows with no window value now sort last in a source's table, as a push holds them. Rows that don't pack are {:table, faults} in errors. Upgrade: read held rows by name with Macaw.Data.Table.get/2, and compare a category as a string.
  • New: steps run a chunk at a time (#20, macaw_data#9 §5, spec/04 §1.4). A transform may declare itself a column-native step: its class (class/1), the context it reads (reads/0), the context bound over the whole input (bind/3), the context as it resolves for a chunk (resolve/3) and the chunk's part (chunk/3); Macaw.Data.Operations.Step runs it, and Step.parts/4 lists each chunk's view and step key, the step's identity, its context as resolved for the chunk and the run it reads, for a caller's cache. filter, take and window (without latest) are per-row steps: each chunk is passed through, dropped, cut to a run without copying, or gathered (Table.Chunk.gather/5, every column's slots copied, its id {input id, step key}), and the merge is concatenation. None of them transposes: at 50k rows take n: 100 is about 0.01 ms (was 81 ms), filter about 8 ms (was 93 ms). window keys a chunk wholly inside it without now, drops one outside, and binary-searches a sorted window field (Macaw.Data.Table.sorted/1, a source's window field, kept by every per-row step) to cut only its edge chunks. Macaw.Data.Table.from_chunks/3 takes views and sorted:; Chunk.values/4 and Chunk.at/3 read a step's column. Macaw.Data.Table.from/2 names its columns without merging a row map. The row helpers moved to an internal module, Macaw.Data.Table.Edge, and are still called through Macaw.Data.Table.
  • New: the associative combines (#21, macaw_data#9 §5, spec/04 §1.4, §1.2, §5). A step of class :combine declares combine/3, two runs' partials merged (associative, in chunk order), and finish/3, its table; Step.merge/5 folds a table's partials and Step.output/3 names a combine's output chunk over its inputs. sum, aggregate and bin (counting, or with count, sum, mean, min, max or extent) and window with latest are combines: any chunking and any grouping of their partials give equal tables. An aggregate may give the combining form, merge/3. Breaking (values): sum and mean sum exactly: integers stay an integer, and with a float among them the exact sum is rounded once to the nearest float (ties to even), so any order of the readings gives the same bits; a float sum may differ from the sequential sum in the last place (e.g. [1.0e16, 1.0, -1.0e16] sums to 1.0, was 0.0). latest's ties are broken by chunk order, then row order, as before. Aggregates without the combining form (median, quantile, distinct, variance, deviation, slope, delta) still read every reading at once; a quantile stays exact, never a sketch.
  • Breaking: lttb runs MinMaxLTTB, and m4 and lttb reduce on one absolute bucket ladder (#22, macaw_data#9 §5, spec/04 §4.4, §4.5, §4.5.1). Buckets are 2^k wide from origin 0 in the x scale's space, named by the new context keys scale (:linear, :time in microseconds, :log as log2 x; :time by default over a time column) and level (the previous level, for a hysteresis of a factor of 2). A chunk's partial depends only on its rows, the scale's kind and the level, so a zoom within a level reuses every partial, a zoom out folds finer partials (Ladder.coarsen/2), buckets straddling a chunk edge and gaps on one merge, and the count option leaves the step key (the new optional callback keyed/1). lttb's tolerance: at most n rows without gaps, the first and last kept, deterministic, the same under every chunking; its mean interpolation error, over the y extent, is at most twice the exact algorithm's plus 0.005 (measured worst 1.27× sines, 1.37× walks, 1.56× sparse spikes). The exact algorithm stays as Macaw.Data.Reduce.lttb/3. m4's tolerance: exact M4 over pixel columns whose boundaries move to a bucket edge, at most a quarter of a pixel away (half under hysteresis); Reduce.m4/3's answer where bucket and pixel edges meet. A budget's reducer (spec/04 §7) picks as the steps do. A time column reduced under the default :time scale buckets in microseconds; pass scale: :linear for the former seconds.
  • New: the global steps and the reshapes (#23, macaw_data#9 §5, spec/04 §1.4). A step of class :global gets every chunk's part in its finish/3: sort, stack, spectrum, and aggregate and bin with an fn that has no combining form (median, quantile, distinct, variance, deviation, slope, delta). A global output is one chunk named by its step key over its inputs' ids and versions, so a step after it still caches; rows kept across chunks are gathered by Step.gather/5, codes remapped. fold is a per-row reshape ({:reshape, indexes, added}, Table.Chunk.reshape/7: carried columns gathered with repeated indexes, the key and value added), and stack gathers across chunks with repeated indexes. Stack.series/2 stacks data as given. No table operation calls Macaw.Data.Table.rows/1 or Operations.table/2 any more; the row helpers Transforms.answer/4, point_reader/2 and column/2 are gone.
  • New: the caching contract (#24, macaw_data#9 §6, spec/04 §1.5, spec/01 §1.1.4). Macaw.Data.Operations.Cache is the example caller: a plain value the caller holds, keyed by {Chunk.id/1, Chunk.version/1, step key}. Cache.run/5 reuses a held part, updates a changed chunk's part by inverse from its base's part where exact (Step.update/4, a step's new optional update/4 and an aggregate's retract/3: counts, bin counts and integer sums), computes any other, and merges; its result equals Step.run/4's. Cache.prune/2 evicts by logical id. A lookup hashes nothing. A chunk with a delta keeps its base's version (Chunk.base_version/1) and a read view built once per state, so reads through a delta (Chunk.values/4, Chunk.at/3, a gather, window's binary search) no longer apply the delta on every read, and compaction makes the view the base without repacking. Fixed: aggregate, sum, bin and spectrum no longer put the chunk's place (:at, :rows) in their step keys, so a chunk's key holds while other chunks come and go.
  • New: parallel chunks on request (#25, macaw_data#9 §7, spec/04 §1.6, §3.2). parallel: true in the context asks Step.run/4 (so every table operation through Operations.apply/4) and Cache.run/5 to work a step's chunks across the schedulers, one task a part, when the input holds at least two chunks and Step.parallel_rows/0 (16,384) rows; otherwise, and by default, they run inline. The parts reach the merge in chunk order whatever worked them, so a parallel run is bit-identical to the inline one, floating-point sums included. parallel is no step's declared context, so it never enters a step key.
  • Measurement: a synthetic frame loop at 50k rows (#26, macaw_data#9 item 7, spec/04 §1.5): a live window with a chunk's worth of new readings a frame, running take, filter, window → m4, window → lttb and sum by series, recomputed in full and per chunk through Operations.Cache. The suite states the per-chunk cost in reductions (its changed chunks at most a full run over one chunk's readings, its frame at most half the full recompute); mix test --only benchmark prints the wall clock. Measured then, before #39–#54: about 400 ms a frame in full, 190 ms per chunk, of which about 95 ms is the m4 and lttb merges. Results and the native-kernel decision are on macaw_data#9; #54 below gives the release's figures.
  • Faster: m4 and lttb merge their partials in work proportional to the chunks, not the buckets (#39, #40, spec/04 §4.5.1). A partial holds a piece per view, each run's buckets ascending, so Ladder.combine/2 concatenates and reads no bucket; the chunk edges are joined once when the merged partial is read (Ladder.runs/1), and finish/3 folds the buckets into pixel columns (m4) or takes its candidates in input order (lttb) in one pass. Results are unchanged under every chunking and grouping. In #26's frame loop the merges alone went from about 100 ms to about 40 ms a frame, and the per-chunk frame from about 215 ms to about 125 ms. A partial's term changed shape (Ladder.partial/0), so an Operations.Cache built by an earlier version is not to be reused.
  • Faster: reading a column costs its slots, not its bits (#41, #42, spec/01 §1.1.5). An all-valid column decodes without expanding its validity bitmap; a slice cuts a bitmap as one little-endian integer; a gather takes an all-set bitmap without reading a bit; and Chunk.gather/5 and reshape/7 gather a run from the chunk's storage at its offset instead of slicing every column first. Bitmaps live in an internal module, Macaw.Data.Table.Bitmap. Results and digests unchanged.
  • Faster: a chunk's m4/lttb partial is one pass with nothing allocated per row but new extremes (#41, #43, spec/04 §4.5.1). The bucket width is read once a partial, an integer x buckets by a shift (s >>> k, s <<< -k), a number x on a linear or time scale is read without a call, and the open bucket's first, last, least and greatest ride as loop arguments. About half the reductions a row; partials unchanged.
  • Faster: grouping, exact sums and filter without per-row overhead (#41, #44, spec/04 §4.1, §4.10, §5). A chunk's groups are found in one pass, one map update a row, and grouping by one category column reads its dictionary codes and names each group once. Exact sums floats' signed mantissas by exponent and counts them in units of 2^-1074 once, when the value is read. filter scans a packed number column's buffer for an ordering against a number; its passing rows, when they are one run, are now cut from the chunk without copying rather than gathered (spec/04 §4.1), so such an output chunk is the input's, read from a row. Values unchanged.
  • Measurement: the frame loop's pure-Elixir cost is pinned (#41, #45, spec/04 §1.5): a per-chunk frame at most 1,100,000 reductions (1,982,948 before #41) and its merges alone 600,000. spec/04 §1.5 states that a frame's garbage collection is the caller's to size: at the default heap a frame's wall time is mostly collecting the cache's partials, and the benchmark now also reports frames in a process whose minimum heap holds the cache (#46 takes up keeping partials off the heap).
  • New: optional native kernels, built and loaded (#48, #38, macaw_data#9 §8, spec/04 §1.7). Macaw.Data.Native compiles Zig kernels through Zigler 0.16.0 when the build has Zig 0.16.0 (.tool-versions), found through zig env; Zigler is an optional dependency, so an adopter that wants the kernels lists {:zigler, "~> 0.16.0", runtime: false}. Without the toolchain the build prints a warning, not a compiler diagnostic, and every step runs its Elixir path. Native.loaded?/0 is decided once at the application's start (the new application module, Macaw.Data.Application) and held in :persistent_term. config :macaw_data, native: :optional | :required, default :optional: :required fails the build without the toolchain, and the start when the kernels don't load. A call over at most Native.sync_rows/0 rows (8,192, #49) runs on a normal scheduler, a larger one on a dirty CPU scheduler. Kernels are built ReleaseSafe, so a panic is never silent. CI tests with Zig under :required, without Zig, and checks that :required without Zig fails the build. ex_doc is now a dev and test dependency, pinned by Zigler's zig_doc to 0.39.1. #48 itself gives no step a kernel; #49–#53 below do. A kernel never replaces its step's Elixir path, which stays the reference, the path for every form a kernel hands back, and the whole of a build without Zig.

  • Faster: a chunk's m4/lttb ladder partial runs in a native kernel where the kernels are loaded (#49, #38, spec/04 §4.5.2). Ladder.partial/4 hands the view to it when x is int64, float64, a UTC, zoned or naive time or a date, and y is int64 or float64, on a linear or time scale; the kernel reads the packed buffers and validity bitmaps in place and answers the Elixir walk's piece term for term (Ladder.partial_elixir/4, the reference). Where the walk would answer past 64 bits or raise, it hands the view back. Native.sync_rows/0 is 8,192, one canonical chunk: a call's time follows its answer's terms, at worst one bucket a row. Reductions count no kernel's work, so Native.elixir/1 runs a function on the Elixir path (Native.active?/0 false inside it), and the frame loop's reduction statements and pins are measured there.
  • Faster: a ladder partial's piece is one packed binary, off the process heap (#50, #38, #46, spec/04 §4.5.3). The Elixir walk packs its segments (Ladder.Packed: a header with the points' number types, a fixed-width head a segment, 88 bytes a bucket; the external term format for a piece of mixed or past-int64 values), and the native kernel writes the same bytes rather than building a term a bucket. The frame loop's cache went from about 824,000 words on the caller's heap to about 11,000, so a collection no longer copies it; a test holds it under 50,000. combine/2 still only concatenates; runs/1 reads each piece once. Results unchanged. A partial's shape changed again (Ladder.piece/0 is {view, binary}), so an Operations.Cache built by an earlier version is not to be reused.
  • Faster: m4's and lttb's finishes run in native kernels where the kernels are loaded (#51, #38, spec/04 §4.5.4). m4_finish and lttb_finish read the merged partial's packed pieces in place, join them as Ladder.runs/1 does, and answer the refs to gather: m4's pixel columns, and lttb's candidates through Reduce.lttb/3. Their arithmetic is Erlang's, operation for operation (exact integers, an integer converted to the nearest float where it meets one, strict floats), so the refs are the Elixir path's (M4.refs/3, Lttb.refs/2, held equal by a property test). A piece kept as a term, a value past int64 or anything Erlang would raise on is handed back to the Elixir path. A finish over at most Native.finish_buckets/0 (2,048) buckets runs on a normal scheduler, a larger one dirty.
  • Faster: the exact sum by group runs in a native kernel where the kernels are loaded (#52, #38, spec/04 §5.1). sum, and aggregate with sum, mean or count, over an int64 or float64 column grouped by one dictionary column or not at all, hand a chunk's view to group_sum, which reads the values, the codes and both bitmaps in place and answers each group's code, first row, rows, readings and exact-sum state (the internal Macaw.Data.Exact) (integers exact in 128 bits, past int64 as a bignum; floats' mantissas summed by biased exponent), so the part is the Elixir fold's term for term (held equal by a property test). A code past the dictionary or a non-finite float is handed back. A call answering more than Native.sync_entries/0 (2,048) groups and exponent sums answers :dirty before building a term and runs on a dirty scheduler. Sum by series over an 8,192-row chunk: 127,254 → 173 reductions, ~2.3 ms → ~80 µs. kernels.zig is split by kernel family (buffers, ladder, finish, group).
  • Faster: filter's test and the gather of the rows it keeps run in native kernels where the kernels are loaded (#53, #38, spec/04 §4.1.1). filter_mask reads the tested column in place and answers the selection itself: present/absent from any column's validity bitmap, an int64 or float64 column against a number compared exactly as Erlang compares (an integer against a float past 2^53 too), a dictionary of strings by each string's answer decided once with the Elixir path's own test. Column.gather/2 (every gather and reshape) copies fixed-width slots (take) and bitmap bits (take_bits) natively, the same bytes. Held equal to the Elixir path by property tests over every form, test and kind of value. v > 0 over an 8,192-row chunk with rows apart: the mask 62,275 → 30 reductions (~470 → ~75 µs), the whole step 112,003 → 6,507 reductions (~1.2 → ~0.5 ms).
  • Measurement: #26's frame loop with every native kernel (#54, #38, #9, spec/04 §1.5.1). Per chunk 958,680 → ~181,000 reductions and 36–47 → 7–10 ms at the default heap (13–16 → 6–8 ms with the frame heap sized); recomputed in full 1,931,845 → ~186,000 reductions, 55–58 → 9–11 ms. With the kernels the frame meets visualize's ~17 ms budget at the default heap, per chunk and in full. A new default-suite pin holds the frame with the kernels to at most 250,000 reductions per chunk, merges alone and in full.
  • New: Macaw.Data.Native.normal_schedulers/1 (#60, #61, spec/04 §1.7). It runs a function with every kernel call in its process on a normal scheduler, whatever the call's size. A call on a dirty scheduler is charged reductions by its time, about four a microsecond, so a count with one in it moves with the host's load. A normal-scheduler call is charged a fixed few. Use it for a statement of work in reductions with the kernels, such as the frame's (spec/04 §1.5.1), never for serving: a large call then holds its scheduler past the 1 ms rule. scheduler/1, finish_scheduler/1 and the new entry_limit/0 answer for it.
  • Fixed: a bounded call is hung only when it is blocked (#33, #62, #67, spec/02 §6.4, spec/01 §9). The conformance kit's runner and the derived engine's sandbox called any call still running at timeout hung, so on a loaded host a slow compile or a capped CPU-bound call was misjudged {:hung, …}. Now a call with a reductions cap is judged by its count while it works. It is hung only once it has been waiting in receive, its reductions unchanged, for timeout ms on end, and each piece of work gives it the whole bound again. A call with no cap still has only the clock. A wait on another process is a block all the same, so give a stored function a generous timeout:, as spec/02 §6.4 asks.
  • Fixed: the first stored function no longer waits on the code server (#69, spec/02 §6.3). In interactive mode a node's first compile loads Elixir's own modules on first use, and a busy code server could hold a bounded compile past its timeout. The application now compiles and runs one trivial function through the sandbox as it starts, so those modules are loaded before any bounded compile. Starting the application costs that one compile.
  • Docs: spec/00 names the dependencies as they are, with no pinned version: macaw_layers required, and table, nx and zigler optional (#34, #70). Every doc reference resolves, and CI builds the docs with --warnings-as-errors.
  • Tests: the suite is warnings-clean and stable under load (#37, #55, #56, #58, #63, #64, #65): call traces run in sessions of their own, test files compile with --warnings-as-errors, and no test's verdict depends on the host's load or on what other tests run beside it. Fifty loaded full-suite runs, with and without Zig, confirm it.

v0.7.0 (2026-10-30)

Terms are the form (dco-tek/macaw/macaw#188), macaw_data's part. A value's identity and validity are the BEAM term's own; JSON and Arrow are adapters at the edge. On macaw_layers 0.3.0. This release is breaking throughout; the guide below is what an adopter of 0.6.1 changes, and the entries after it are the detail.

Upgrading from 0.6.1

  1. Depend on macaw_layers ~> 0.3.0 (#27); its own upgrade guide applies to any kernel call you make directly.
  2. Every schema version changes (D1). A stored version, cache key or golden written by 0.6.1 differs; regenerate it. A schema's names, states and roles are now strings in the struct (Schema.new/1 normalizes them), so code that pattern-matches :latency inside a schema matches "latency"; every function that takes a name still takes either spelling.
  3. A data color is a color term (D2): a %Macaw.Layers.Color{} literal or a typed color, never a string. Parse CSS text at your input edge with Macaw.Layers.Color.parse!/1, or write ~RGB"#3b6fa8" in code. A role spelled like a CSS name ("orange") is now a valid status role.
  4. A derived source is built from its value (D3). Pass Derived.new/1 a keyword list or a map of terms; read a stored form with Derived.from_map/2 or from_json/2. to_map/1 and to_json/1 now answer {:ok, form} or faults.
  5. The operations registry's digest changes (D4); a compile cache keyed by it recompiles once. Operations.step_digest/2 is new: a checked step's canonical identity.
  6. Category values read back as strings (D5): a :category column holding :ok answers "ok" from rows/1, get/2 and column/2; compare with strings, or with Operations.same?/2. nil, true and false are kept.
  7. Packed columns are faithful and Arrow-compatible (D5). A mixed integer/float column and a zoned time read back exactly as given; anything that held packed buffers (stored binaries) changes with the bitmap order.

Changes

  • Breaking: on macaw_layers ~> 0.3.0 (#27): the kernel's digest is over the value, so every pinned version moved before D1 changed what it digests.
  • Breaking: every schema version changes (D1, #28, dco-tek/macaw/macaw#188). Schema.version is the kernel's digest of the schema's term, not of its JSON form, which is now only an adapter at the edge; a value entry's color no longer goes through the JSON form when Schema.new/1 versions the schema, so a typed color there can't crash it. The struct holds every field name, state and role as its string, normalized once by new/1, so fields, identity/1, window_field/1, compatibility/2's faults and identity/2's maps name fields by their strings; every function that takes a name still takes either spelling. A state named in both spellings in one description is :duplicate.
  • Breaking: data colors are color terms, and a string is never a color (D2, #29, dco-tek/macaw/macaw#187, dco-tek/macaw/macaw#188). A value entry's color, a :color field's row values (a derived source's output), a :color parameter and Schema.color?/1 take a literal %Macaw.Layers.Color{} or a theme-relative typed color ({:color, "good"}, {:alpha, {:color, "bad"}, 0.74}), its names in a theme's name grammar; a string there, "#2e7d32" or "red", is {:type, :color}. new/1 holds a typed color's names as strings, so {:color, :good} and {:color, "good"} are one version. The schema's JSON form writes and reads an entry's color in macaw_layers' color form (Macaw.Layers.Color.Form: {"$rgb": "#2e7d32"}, {"$color": "good"}), making no atom. Macaw.Data.Table.column_types/1 infers :color for a column of color terms; a column of strings stays :text. A status role spelled like a CSS name (:orange) is now a role. Upgrade: read CSS text into a color at your input edge with Macaw.Layers.Color.parse!/1, or write ~RGB"#2e7d32" in source (import Macaw.Layers.Color, only: [sigil_RGB: 2]); a stored schema whose entry colors are strings reads back refused, so convert its colors with parse!/1 before reading it.
  • Breaking: a derived source is built from its value, and its forms are an adapter (D3, #30, dco-tek/macaw/macaw#188). Derived.new/1 takes the value only, a keyword list or a map of its fields as terms; it no longer guesses whether it was given a map form, and a map form's string keys are each :unknown_key. The map and JSON forms are written and read by macaw_layers' JSON adapter (Macaw.Layers.Form), with the kernel's tags and the source's own in one tag table: to_map/1 and to_json/1 answer {:ok, form} or every value without a form, by path; from_map/2 and from_json/2 read through the gates (the limits, the version, the tag table, the keys, making no atom) and then judge the value with new/1. The output schema is used as built, a %Schema{} as given and fields by Schema.new/1, never written to its form and read back; in the form it is Schema.to_map/1's. Derived.Form's private tag scheme is gone. A color, literal or typed, in a schema's value entry reaches a stored function's context as the term it is; before, its tags were respelled as strings. Upgrade: the form changed: an input is {"$input": [name, window]}, a source's name {"$source": name}, per {"$atom": "identity"} or {"$identity": input}, a region an object keyed by field names, and to_json/1 stamps "$version": 1. Nothing in this library stored the old form of pair lists, so no version step reads it: write a stored source again with to_map/1 from its value. Read a map form with from_map/2, not new/1.
  • Breaking: the operations registry's digest/1 is the kernel's term digest of its names and modules, not a SHA-256 of term_to_binary (D4, #31, dco-tek/macaw/macaw#188), so every registry digest changes and a compile cache keyed by one recompiles once. New: Operations.step_digest/2, a checked step's canonical identity, for caches of a step's results.
  • Breaking: a table's packed columns are faithful and laid out as Arrow lays them out (D5, #16, dco-tek/macaw/macaw#188), spec/01 §1.1. Packing never changes a value. A category column of strings and atoms is one Arrow dictionary of strings, so rows/1, get/2 and column/2 read a category's atom back as the string that spells it (:ok and "ok" are one code, read as "ok"), in every form the column takes; nil, true and false stay themselves, beside the names in side bitmaps, so [:ok, true, "warn"] reads back ["ok", true, "warn"]. Validity bitmaps are least-significant-bit first, Arrow's order, not Elixir's <<b::1>> order, so every stored packed binary changes. A DateTime keeps its zone (name, abbreviation and both offsets, in a per-column dictionary, with no time-zone database) and its precision, so a time reads back as given rather than in UTC at microsecond precision. A text column is an offsets buffer and one data buffer, Arrow utf8 when every value is valid UTF-8 and Arrow binary when one is not; a category column holding a string that is not valid UTF-8 is kept as its list. new/1 takes NaiveDateTimes and Dates in a time column as themselves, and refuses a value in a :color column that is not a color term (Schema.color?/1), {[name, i], {:type, :color}}. Upgrade: compare a category value with Operations.same?/2, or as a string.
  • Fixed: a number column mixing integers and floats reads back each as given, through an integral bitmap, or a dense union of int64 and float64 when an integer is beyond ±2^53; an integer beyond int64 keeps its column a list rather than becoming a float. A time column keeps each value's zone and precision, so ~U[2026-01-01 00:00:00Z] and ~U[2026-01-01 00:00:00.000000Z] stay distinct (#16, dco-tek/macaw/macaw#188).

v0.6.1 (2026-10-29)

  • Fixed: compiling macaw_data in an app without the optional table package warned that Table.Reader.impl_for/1 is undefined; CI now compiles it without its optional dependencies (#7).

v0.6.0 (2026-10-29)

  • What a source steps its window by: Window.before/4, region/2 (visualize's box/2: a window as the region it asks of a schema), range/2, and check/2 of the axes alone as well as of a request (dco-tek/macaw/macaw#185, #5).
  • Packing a table and reading it back do about half the work: types inferred only where not given, values checked in one pass, column names merged once, rows built a column step at a time, and time columns converted by the date's fields, a fifth of DateTime's own conversion (dco-tek/macaw/macaw#185, #4).
  • Breaking: a delivery keeps the rows its source gave (dco-tek/macaw/macaw#185). %Delivery{rows, names, types, served, schema, origin}: nothing is packed or rewritten when it is made; Delivery.rows/1 and Macaw.Data.Table.rows/1 of a delivery read the rows as they are (a row list uncopied, tuple rows unchanged); Delivery.table/2 and table!/2 pack them for a reader that wants columns; named_rows/1 names tuple rows and tensors by names:; empty?/1. The table field is gone: read Delivery.table!/1. Finishing and push keep the rows' form. Macaw.Data.Table.unnamed?/1 (#3).

v0.5.4 (2026-10-28)

  • The macaw logo in the docs and the README (macaw#175).
  • Tests check "makes no atom" on the input's own names, never the node's atom count, and the atom gate forbids the count in test/ (#1).

v0.5.3 (2026-10-28)

  • On Hex: depends on macaw_layers ~> 0.2.2 from Hex instead of its git tag, and the README installs from Hex (macaw#155).
  • The README opens with the early-release notice (macaw#174).

v0.5.2 (2026-10-28)

  • Dependencies named by their dco-tek/macaw group paths (macaw#152).
  • Package metadata for Hex: the public home, the changelog link, the files shipped, and docs with ex_doc (macaw_compose#155).

v0.5.1 (2026-10-28)

  • macaw_data v0.5.1: on macaw_layers v0.2.0 (macaw_compose#145)

v0.5.0 (2026-10-27)

  • macaw_data v0.5.0: transforms, table operations and aggregates (macaw_compose#83): the behaviors, the registry, twelve table operations, thirteen statistics, Macaw.Data.Ops in the sandbox, budget reducers by name

v0.4.0 (2026-10-27)

  • macaw_data v0.4.0: derived sources (macaw_compose#82): the model, the sandbox and its gates, host functions and the engine

v0.3.0 (2026-10-27)

  • macaw_data v0.3.0: the source contract (macaw_compose#58): the behavior, description, request and parameters; windows and finishing; push; the conformance kit

v0.2.0 (2026-10-27)

  • macaw_data v0.2.0: the data model (macaw_compose#57): units and quantities, the columnar table, the schema, the delivery

v0.1.0 (2026-10-27)

  • macaw_data 0.1.0: Macaw.Data.Event