Macaw.Data.Source.Combined (macaw_data v0.18.0)
View SourceOne source over several, routed by window (spec/01 §10.2): live data and its history as one source, say an in-process telemetry source holding the last hour in front of a store holding everything.
The parts are Macaw.Data.Sources, listed freshest first, each a source (described with
no context) or {source, context} (described with its own: fetch/2 takes no context,
so each part's is held here). A fetch asks the first part for the window; what it served
starts at its delivery's served window (else its earliest row), and the part of the
window before that goes to the next part, and so on. Each later part's rows are trimmed
to the half-open window it was asked for, [from, served), so a row at a seam comes from
the fresher part alone, and keeps only rows at least the first part's resolution before
served, so no two rows lie closer than the resolution the union states, whatever grid
each part's rows lie on (macaw_data#282); a part serving from after the window's end
hands on that end, whole. The schema must have a :time or :number window field to
route by.
The combination's schema is the first part's, whole: its version, labels and facts.
Every later part's schema must be compatible with it (Macaw.Data.Schema.compatibility/2)
and window over the same field. Its versions may differ, as a process's live source and a
store's history always do. A later part's rows are read in the first schema's words
before routing: keyed by the first part's spellings, numbers converted into its units,
and fields only the later schema describes dropped. The first part's rows are never
touched.
It is described by its parts (spec/01 §2.1, §10.2): the first part's parameters,
lateness, idle_after and chunk_span, and viewer independent only when every part
is. Every later part is asked the first's parameters, so it must declare each, taking
every value the first's takes; otherwise describe and fetch answer {:error, {:combined_params, faults}}, each {position, name, :missing | {:narrower, first, later}}.
A combination honors :window, pushes when its first part does (its subscription is the
first's, its pushes pass through unchanged), and honors :as_of when every part does.
Its request goes to every part with only the window routed, and its subscription to
the first alone. What finishing does over the union no part is asked: a region,
latest, an order, a budget and its reducer (macaw_data#273), since a part's own cut
is no cut of the union. as_of, params, viewer and now reach every part as they
are.
Its delivery states what the parts said of their answers (spec/01 §10.2): stale
when any part asked answered stale; the parts' notes, each once, the freshest first,
joined by "; "; the first part's params, position and resolution; capped when
the oldest part asked stops short of the window or says it cut; and complete_until the
least claim of the parts asked, none when any claims none, but the first part's for a
snapshot taken for a subscription. reduced_by, reduced_with, ordered and origin
are none: no part is asked a budget, a reducer or an order, finishing states them of the
union, and a part's origin names that part.
No later part's rows are finer than the first part's resolution, the union's: where
it has a spacing (a number, or a name such as :minute over a :time field), a later
part keeping rows at a finer resolution, read in the first's units, or at none, is
{:error, {:combined_resolution, position, theirs, first}}. A coarser history combines.
Parts that deliver tables are concatenated at the seam as tables: each kept run of a
part's table is a slice of it, and the combination's rows are one Macaw.Data.Table of
their chunks, copied nowhere, when every part kept a table of the same columns. A part
whose rows are rows, or are read in the first schema's words, makes the combination's
rows rows.
Ported from visualize's Visualize.Feed.Combine.
Examples
iex> alias Macaw.Data.Source
iex> alias Macaw.Data.Source.{Combined, Signal}
iex> live = Signal.new!(:discrete, [:t, :n], tick: 9, limit: 3)
iex> history = Signal.new!(:discrete, [:t, :n], tick: 9)
iex> {:ok, combined} = Combined.new([live, history])
iex> {:ok, description} = Source.describe(combined, nil)
iex> request = Source.Request.new!(window: [2, 9], now: ~U[2026-11-01 00:00:00Z])
iex> {:ok, delivery} = Source.fetch(combined, description, request)
iex> {Macaw.Data.Table.column(delivery.rows, "t"), delivery.served.window, delivery.served.capped}
{[2, 3, 4, 5, 6, 7, 8, 9], [2, 9], false}
iex> Macaw.Data.Source.Combined.new([])
{:error, [{[], {:type, :parts}}]}
Summary
Functions
A reason's words (spec/00 §5): this module's, else the library's.
One source over several, freshest first, each a source or {source, context}:
{:ok, combined}, or every fault (spec/00 §5): {[], {:type, :parts}} for anything but
a non-empty list, {[i], {:type, :part}} for an element that is no struct nor
{struct, context}, and {[key], :unknown_key} for an option it does not take (it takes
none yet). Whether each part is a source is its describe's to say.
As new/2, the combination alone, raising Macaw.Data.Error naming the faults.
Types
@type part() :: Macaw.Data.Source.t() | {Macaw.Data.Source.t(), term()}
A part: a source, described with no context, or a source and its context.
@type t() :: %Macaw.Data.Source.Combined{ parts: [{Macaw.Data.Source.t(), term()}, ...] }
Functions
A reason's words (spec/00 §5): this module's, else the library's.
One source over several, freshest first, each a source or {source, context}:
{:ok, combined}, or every fault (spec/00 §5): {[], {:type, :parts}} for anything but
a non-empty list, {[i], {:type, :part}} for an element that is no struct nor
{struct, context}, and {[key], :unknown_key} for an option it does not take (it takes
none yet). Whether each part is a source is its describe's to say.
As new/2, the combination alone, raising Macaw.Data.Error naming the faults.