Macaw.Data.Source.Combined (macaw_data v0.18.0)

View Source

One source over several, routed by window (spec/01 §10.2): live data and its history as one source, say an in-process telemetry source holding the last hour in front of a store holding everything.

The parts are Macaw.Data.Sources, listed freshest first, each a source (described with no context) or {source, context} (described with its own: fetch/2 takes no context, so each part's is held here). A fetch asks the first part for the window; what it served starts at its delivery's served window (else its earliest row), and the part of the window before that goes to the next part, and so on. Each later part's rows are trimmed to the half-open window it was asked for, [from, served), so a row at a seam comes from the fresher part alone, and keeps only rows at least the first part's resolution before served, so no two rows lie closer than the resolution the union states, whatever grid each part's rows lie on (macaw_data#282); a part serving from after the window's end hands on that end, whole. The schema must have a :time or :number window field to route by.

The combination's schema is the first part's, whole: its version, labels and facts. Every later part's schema must be compatible with it (Macaw.Data.Schema.compatibility/2) and window over the same field. Its versions may differ, as a process's live source and a store's history always do. A later part's rows are read in the first schema's words before routing: keyed by the first part's spellings, numbers converted into its units, and fields only the later schema describes dropped. The first part's rows are never touched.

It is described by its parts (spec/01 §2.1, §10.2): the first part's parameters, lateness, idle_after and chunk_span, and viewer independent only when every part is. Every later part is asked the first's parameters, so it must declare each, taking every value the first's takes; otherwise describe and fetch answer {:error, {:combined_params, faults}}, each {position, name, :missing | {:narrower, first, later}}.

A combination honors :window, pushes when its first part does (its subscription is the first's, its pushes pass through unchanged), and honors :as_of when every part does. Its request goes to every part with only the window routed, and its subscription to the first alone. What finishing does over the union no part is asked: a region, latest, an order, a budget and its reducer (macaw_data#273), since a part's own cut is no cut of the union. as_of, params, viewer and now reach every part as they are.

Its delivery states what the parts said of their answers (spec/01 §10.2): stale when any part asked answered stale; the parts' notes, each once, the freshest first, joined by "; "; the first part's params, position and resolution; capped when the oldest part asked stops short of the window or says it cut; and complete_until the least claim of the parts asked, none when any claims none, but the first part's for a snapshot taken for a subscription. reduced_by, reduced_with, ordered and origin are none: no part is asked a budget, a reducer or an order, finishing states them of the union, and a part's origin names that part.

No later part's rows are finer than the first part's resolution, the union's: where it has a spacing (a number, or a name such as :minute over a :time field), a later part keeping rows at a finer resolution, read in the first's units, or at none, is {:error, {:combined_resolution, position, theirs, first}}. A coarser history combines.

Parts that deliver tables are concatenated at the seam as tables: each kept run of a part's table is a slice of it, and the combination's rows are one Macaw.Data.Table of their chunks, copied nowhere, when every part kept a table of the same columns. A part whose rows are rows, or are read in the first schema's words, makes the combination's rows rows.

Ported from visualize's Visualize.Feed.Combine.

Examples

iex> alias Macaw.Data.Source
iex> alias Macaw.Data.Source.{Combined, Signal}
iex> live = Signal.new!(:discrete, [:t, :n], tick: 9, limit: 3)
iex> history = Signal.new!(:discrete, [:t, :n], tick: 9)
iex> {:ok, combined} = Combined.new([live, history])
iex> {:ok, description} = Source.describe(combined, nil)
iex> request = Source.Request.new!(window: [2, 9], now: ~U[2026-11-01 00:00:00Z])
iex> {:ok, delivery} = Source.fetch(combined, description, request)
iex> {Macaw.Data.Table.column(delivery.rows, "t"), delivery.served.window, delivery.served.capped}
{[2, 3, 4, 5, 6, 7, 8, 9], [2, 9], false}

iex> Macaw.Data.Source.Combined.new([])
{:error, [{[], {:type, :parts}}]}

Summary

Types

A part: a source, described with no context, or a source and its context.

t()

Functions

A reason's words (spec/00 §5): this module's, else the library's.

One source over several, freshest first, each a source or {source, context}: {:ok, combined}, or every fault (spec/00 §5): {[], {:type, :parts}} for anything but a non-empty list, {[i], {:type, :part}} for an element that is no struct nor {struct, context}, and {[key], :unknown_key} for an option it does not take (it takes none yet). Whether each part is a source is its describe's to say.

As new/2, the combination alone, raising Macaw.Data.Error naming the faults.

Types

part()

@type part() :: Macaw.Data.Source.t() | {Macaw.Data.Source.t(), term()}

A part: a source, described with no context, or a source and its context.

t()

@type t() :: %Macaw.Data.Source.Combined{
  parts: [{Macaw.Data.Source.t(), term()}, ...]
}

Functions

format_error(reason)

@spec format_error(term()) :: String.t()

A reason's words (spec/00 §5): this module's, else the library's.

new(parts, opts \\ [])

@spec new(
  term(),
  keyword()
) :: {:ok, t()} | {:error, [{list(), term()}, ...]}

One source over several, freshest first, each a source or {source, context}: {:ok, combined}, or every fault (spec/00 §5): {[], {:type, :parts}} for anything but a non-empty list, {[i], {:type, :part}} for an element that is no struct nor {struct, context}, and {[key], :unknown_key} for an option it does not take (it takes none yet). Whether each part is a source is its describe's to say.

new!(parts, opts \\ [])

@spec new!(
  term(),
  keyword()
) :: t()

As new/2, the combination alone, raising Macaw.Data.Error naming the faults.