ReactiveDag.Node.Poll (reactive_dag v0.17.0-rc.40)

Copy Markdown View Source

This node's rows come from OUTSIDE the graph: poll MuniWatch.Crawler.

A source is an ordinary node whose rows a scanner writes rather than a combinator computing them. ReactiveDag.Source is the behaviour that fetches them, and it deliberately runs outside the drain — external I/O must not sit inside a depth-ordered recompute.

Everything downstream is then an ORDINARY EDGE. A leaf holding one kind of what the crawl found reads the source like any other input:

# the source
reactive do
  id :agenda_center
  poll MuniWatch.Sources.AgendaCenter, every: "0 12 * * *", args: [recent: true]
end

# a consumer of it
reactive do
  id :agenda_docs
  reduce over: :agenda_center, group_by: :key, expand: &agenda_only/2
end

This replaced scan Mod on each fed leaf plus leaf_cells/1 on the module plus verify_scan!/3 to police the two agreeing — the same fact written twice, with a verifier for when the copies drifted. The cells a source feeds are now plan.parents[id], which cannot disagree with anything.

every: and args: belong here rather than on the fed leaves, because they describe the POLL: one crawl has one cadence and one bound. Spread across leaves they had to be reassembled, and a source feeding two leaves could silently lose the args one of them declared.