The address reaper: reap/2, the plain function that removes address
rows whose execution finished longer ago than their horizon (ADR-0002,
sections 5 and 6).
The two handles it works on
reap/2 takes the router's configuration as the storage it works on,
and the configuration carries both handles it needs:
- the host's
:repo, with the:table_prefixand:prefixthat name the address table in it (StatifierRouter.Config.table/2): the rows it reads, stamps and deletes; - the
:store, the%StatifierPersistence.Storage{}the executions are kept in: the only thing it reads there is an execution's status, throughStatifierPersistence.Storage.fetch_execution/2.
A configuration without a :store is refused by the function head. The
bindings are the second argument, the host's current ones, and not the
configuration's: the horizon is computed from the bindings handed to each
reap.
What one reap does
For each address row it examines, in the order of the rows' ids:
- a row already stamped
terminal_seen_atis not read again: a terminal execution stays terminal; - for a row not yet stamped, it reads the execution's status. An
:activeexecution leaves the row as it is. A terminal one (:completed,:failedor:cancelled) is seen terminal now, and its row is stamped with the reap's time unless the next rule deletes it in the same reap; - a row whose execution is terminal is deleted once its horizon has
elapsed since
terminal_seen_at: whenterminal_seen_atplus the horizon is not later than the reap's time.
The horizon of a row is the longest dedupe horizon, horizon_ms, of any
enabled binding among bindings whose document is the row's document.
A document no enabled binding names has a horizon of zero. So disabling
or removing every binding for a document frees the rows of its finished
executions at the next reap, including a row that reap is the first to
see terminal; a later event for such an address opens a fresh execution
under :if_absent.
It deletes address rows only. It never deletes, alters or steps an execution, and it never writes the input log.
Its cost is bounded
Every row not yet stamped costs one status read, so a reap over every
row would grow with the number of live addresses. One call examines at
most :limit rows (1000 by default), those whose id is greater than
:after, and answers with next: the id of the last row it examined
when it examined a full :limit of them, and nil when it reached the
end of the table. A host sweeps the whole table by calling again with
after: next until next is nil. A host that only ever calls reap/2
with no options examines the first :limit rows each time, which frees
nothing behind them.
An {:error, reason} from fetch_execution/2 for any examined row ends
the call before it writes anything, and is returned.
This package runs no process to call it: the host schedules it, as it
schedules StatifierRouter.Dedupe.reap/2 (ADR-0002, section 6).
Summary
Types
What one reap did: how many rows it stamped terminal_seen_at on, how
many it deleted, and the cursor to continue from, or nil at the end of
the table.
Functions
Stamps and deletes the address rows under config as the module
documentation describes, with each row's horizon computed from
bindings.
Types
@type result() :: %{ stamped: non_neg_integer(), deleted: non_neg_integer(), next: pos_integer() | nil }
What one reap did: how many rows it stamped terminal_seen_at on, how
many it deleted, and the cursor to continue from, or nil at the end of
the table.
Functions
@spec reap(StatifierRouter.Config.t(), [StatifierRouter.Binding.t()], keyword()) :: {:ok, result()} | {:error, term()}
Stamps and deletes the address rows under config as the module
documentation describes, with each row's horizon computed from
bindings.
opts:
:now- aDateTimein UTC, the reap's time. Defaults toDateTime.utc_now/0.:limit- a positive integer, the most rows this call examines. Defaults to 1000.:after-nilor a positive integer: examine only rows whose id is greater. Defaults tonil, the start of the table.
Returns {:ok, result}, {:error, reason} from
StatifierPersistence.Storage.fetch_execution/2, or
{:error, reason} for a malformed option ({:invalid_opts, opts},
{:unknown_key, name}, {:invalid_value, name, value}).