Keyed tumbling-window aggregation — the Phase 5 output-commit instrument (§6.4).
This is the first model that implements commit/2, and its purpose is to make the
output-commit quarantine load-bearing and proven rather than dead-but-sound. It
is deliberately the maximally partitionable workload — one independent key per LP,
no cross-key dependency — which is exactly the case §11.5 says to just
data-parallelize. So it is the mechanism instrument, not the research workload:
its value is a clean oracle, a tunable per-event cost, tunable input skew, and a
concrete irreversible effect sitting on the far side of the GVT horizon.
What it computes
Each LP owns one key (its lp_id). An input {:input, value} at event-time at
folds into tumbling window w = div(at, window). Processing the first input of a
window schedules a self-event {:close, w} at (w+1)*window — the window boundary.
That close event is the thing committed once the watermark passes it; commit/2
reads the now-final window accumulator and emits the total exactly once.
Why this exercises the quarantine
Inputs arrive out of order (the whole point of optimistic stream processing). A late
but still-above-GVT input for window w rolls the speculative {:close, w} back and
re-emits it with the corrected total — the OLD speculative close must never have
committed. It commits only once GVT (the watermark) passes (w+1)*window, which by
GVT's definition means no further input for window w can still be in flight (any
such input has at < (w+1)*window ≤ GVT, and nothing below GVT is transient). So:
- W is the commit-latency SLA. A window commits only when GVT clears its close, so GVT-lag is commit latency, and the time window (§11.3) bounds that lag.
- commit/2 reads a final accumulator. When
{:close, w}commits, every window-winput is below GVT and permanent, sowindows[w]is complete and stable.
handle_event/3 is pure (fold + emit only). commit/2 is the single sanctioned
side-effect site: it reports the finalized window to report_to.