ReactiveDag.Node.Fingerprint (reactive_dag v0.17.0-rc.6)

Copy Markdown View Source

The one value that decides whether an observation MOVED.

Two rungs of the ladder need the same question answered, for the same reason: work is expensive, and repeating it over unchanged input is the cost the engine exists to avoid.

  • per_key skips its action when the input fields it depends on are unchanged — the point of the rung, since the action may be an LLM call.
  • a source-fed leaf reports a key as changed only when what it observed moved. A re-crawl that finds identical bytes must not fire the cascade.

Both compare ONE value, not the whole row, and for the same reason: a row carries fields that move on every observation without the observation having changed anything. A last_seen_at changes by definition. An etag can be re-issued for identical bytes. Comparing every attribute reports those as changes and re-runs everything downstream.

The two forms

A field list hashes those fields:

fingerprint [:content_md5, :title]

A function computes the value itself, for when "the same observation" is not a plain field comparison — a normalized URL, a hash of a hash, a version folded into a digest:

fingerprint fn row -> "#{row.content_md5}|#{:erlang.phash2(row.title)}" end

Either way the result is stored on the row (fingerprint_attribute, default :fingerprint) so the next pass has something to compare against.

Why the host decides

What counts as "the same observation" is domain knowledge the library cannot infer. Usually it is the content digest. Deliberately not always: a crawler may fold a listing title into it, so a re-titled document re-fires downstream work even though its bytes are identical. That is a correct domain judgement and the library has no business overriding it — it only needs somewhere to put the answer.

Summary

Types

How a node computes its fingerprint: field list, function, or none.

Functions

The attribute a fingerprint is stored in when the node names none.

The fingerprint of row under spec.

Put value into attrs under attr, raising with the fix when the resource has nowhere to store it.

Types

spec()

@type spec() :: [atom()] | (map() -> term()) | nil

How a node computes its fingerprint: field list, function, or none.

Functions

default_attribute()

@spec default_attribute() :: atom()

The attribute a fingerprint is stored in when the node names none.

of(fun, row)

@spec of(spec(), map()) :: String.t() | nil

The fingerprint of row under spec.

nil means the node declares none — every pass then treats the row as moved, which is the correct default: a node that has not said what makes it stale must not be assumed fresh.

A function form returning nil means the same thing, which is how a source says "I could not determine this" without inventing a value that would read as unchanged.

put(attrs, attr, value, resource, declared_by)

@spec put(map(), atom(), term(), module(), String.t()) :: map()

Put value into attrs under attr, raising with the fix when the resource has nowhere to store it.

A missing column is a configuration mistake that would otherwise be silent and expensive: the value is dropped on write, so the next pass reads nil, compares unequal, and re-runs the work the fingerprint existed to skip. It would look exactly like a fingerprint that never matches.