ReactiveDag.Node.Scan (reactive_dag v0.17.0-rc.2)

Copy Markdown View Source

The SCANNER that feeds this leaf: scan MuniWatch.Crawler.

A leaf's keys come from outside the graph — a crawled website, an S3 listing, an API. ReactiveDag.Source is the behaviour that fetches them, and it deliberately runs in a POLL phase outside the drain (external I/O must not sit inside a depth-ordered recompute). What was missing was the library knowing which scanner feeds which leaf: the binding lived in the scanner's own leaf_cells/1 and in an opaque source :atom label, joined by string matching, with Source.verify!/2 existing to catch the mismatches that arrangement invites.

Declaring it here makes the pairing a fact of the graph. graph/2 verifies it during assembly (a scan naming a module that is not a Source, or a scanner whose leaf_cells/1 disowns this leaf, raises there), and ReactiveDag.Source.poll_all/2 can find every scanner from the plan instead of a hand-kept list that drifts.

This is the SINGLE-LEAF spelling — one scanner, one leaf. A source that feeds MANY leaves (one per discovered kind, or a generator's instances) still implements leaf_cells/1 and is passed to verify!/2 directly: leaf_cells/1 takes the lowered graph precisely because those leaves come from live data, which no compile-time declaration can name.