An instance's shared, lock-free state.
new/1 creates every ETS table and :atomics array an instance needs; the
instance's Limen.Owner calls it when the instance starts, owns the tables,
and publishes them in its Limen.Instance. Requests read and write the
tables directly; they never message a process. Background work (rotating
time windows, sweeping expired entries) lives in Limen.State.Rotator and
Limen.State.Sweeper, one of each per instance.
Tables
- Time windows (
Limen.State.Window): for each of:second,:minuteand:hour, three epoch slots of exact counters plus a Count-Min Sketch (Limen.Sketch.CountMin) per slot for keys that arrive once the slot is full. - GCRA (
Limen.State.Gcra): theoretical arrival times for hard limits. - Bans (
Limen.State.BanList): banned prefixes with their expiry, and an outbox of local ban changesLimen.Clusterbroadcasts. - Distinct counting: a HyperLogLog (
Limen.Sketch.HyperLogLog) of client prefixes per minute epoch, whose estimate a background tick publishes asactive_prefixes/1, and a rotating Bloom filter (Limen.Sketch.RotatingBloom) of (prefix, path) pairs so each client's distinct paths can be counted in its time window. - Crawler verification (
Limen.Signal.Fcrdns): a cache of results and a bounded queue of addresses to verify.
Memory bounds
Every table has a hard cap on the number of keys (see the :state options
in Limen.Config), counted with :atomics so the request path never has to
ask ETS for a table size. Sketches are allocated once and never grow. Once a
table is full:
- windows count new keys in the slot's sketch instead (estimates may overcount, never undercount);
- GCRA stops tracking new keys, which are then not limited;
- the ban list refuses new bans.
Each case increments the :saturated counter in Limen.Stats.
Summary
Functions
Distinct client prefixes instance saw over the last one to two minutes,
as last estimated in the background.
Duration of window in milliseconds.
Memory used by an instance's state, in bytes, by table.
Whether a client prefix had not requested path recently.
Adds a client prefix to the active prefix count.
Clears every table and counter of an instance. Meant for tests.
Number of epoch slots per window.
Window names and their durations in milliseconds.
Types
@type t() :: %{ windows: %{ required(window()) => %{ tables: tuple(), counts: :atomics.atomics_ref(), sketches: tuple(), max_keys: pos_integer() } }, gcra: %{table: :ets.tid(), size: :atomics.atomics_ref()}, bans: %{ table: :ets.tid(), size: :atomics.atomics_ref(), outbox: :ets.tid(), outbox_size: :atomics.atomics_ref() }, distinct: %{ prefixes: tuple(), paths: Limen.Sketch.RotatingBloom.t(), estimate: :atomics.atomics_ref() }, fcrdns: %{ cache: :ets.tid(), pending: :ets.tid(), size: :atomics.atomics_ref() } }
@type window() :: :second | :minute | :hour
Functions
@spec active_prefixes(Limen.Instance.t()) :: non_neg_integer()
Distinct client prefixes instance saw over the last one to two minutes,
as last estimated in the background.
@spec duration(window()) :: pos_integer()
Duration of window in milliseconds.
@spec memory(atom() | Limen.Instance.t()) :: %{required(term()) => non_neg_integer()}
Memory used by an instance's state, in bytes, by table.
@spec new_path?(Limen.Instance.t(), Limen.IP.prefix(), String.t()) :: boolean()
Whether a client prefix had not requested path recently.
@spec observe_prefix(Limen.Instance.t(), Limen.IP.prefix(), integer()) :: :ok
Adds a client prefix to the active prefix count.
Adding a prefix again in the same minute changes nothing, so callers only need to on its first request of the minute.
@spec reset(atom() | Limen.Instance.t()) :: :ok
Clears every table and counter of an instance. Meant for tests.
@spec slots() :: pos_integer()
Number of epoch slots per window.
@spec windows() :: keyword(pos_integer())
Window names and their durations in milliseconds.