Keeps the events table from growing without bound.
Analytics is the one table in a PhoenixKit app that grows with traffic rather than with content, so it needs a story for old data from day one. Once an hour this process:
- Rolls up completed days into
PhoenixKitWebAnalytics.Schemas.DailyStatrows — page views, visitors, sessions, bounces, and total session seconds, per site. - Prunes raw events older than
web_analytics_retention_days(365 by default;0disables pruning), in batches, and only for days that were rolled up first.
So the long-range trend line is permanent while the raw rows behind it are
not. What is lost at the retention horizon is the ability to break an old day
down by page or referrer — see PhoenixKitWebAnalytics.Reports for how
reports handle the boundary.
The watermark
Progress is a single date — "every day up to and including this one is
rolled up" — kept in the web_analytics_rolled_through setting. Each pass
walks forward from it (at most 60 days at a time, so a first run against a
large backlog spreads over several hours), and advances it only past days
whose rollup committed. Prune never deletes past the watermark, and does
nothing at all if the watermark can't be read: a database hiccup must cost
an hour of compaction, never a day of history.
A day is re-rolled by the passes in the first 3 hours after it ends. A hit that was being written as midnight passed lands in "yesterday" after that day's first rollup; a re-roll picks it up, and replacing the day whole makes it idempotent. After that the day is final — re-rolling it every hour would re-read a whole day of raw events 24 times for nothing.
Each breakdown keeps at most 5000 values per day (the most visited), so a site with an id in every URL can't turn one day's rollup into a million rows. The long tail below that still counts in the day's totals.
Sessions from before schema V3 get their first hit marked here too
(backfill_session_starts/0), a batch at a time, so the migration never
rewrites a large table in one statement.
Days are UTC days, bounded with inserted_at ranges — never with a
DATE(...) cast, which would depend on the database session's time zone.
Scheduling
The first run is deliberately a couple of minutes after boot: a host restart should not spend its first seconds deleting rows while it is also serving the post-deploy traffic spike. Runs are skipped entirely while the module is disabled.
Summary
Functions
Marks the first hit of every session recorded before schema V3, which
introduced session_start — at most 100 batches of
5000 sessions per call, walking session_id in order from a
cursor kept in the web_analytics_session_starts_backfilled setting ("done" once finished).
Returns a specification to start this module under a supervisor.
Deletes raw events past the retention horizon, in batches.
The last day known to be fully rolled up, or nil when nothing has been
rolled up yet. :error when the setting can't be read.
Aggregates one day into DailyStat rows, one per site.
Rolls up the completed days past the watermark, and re-rolls the last 2.
Runs a rollup + prune pass immediately, in the caller's process.
Asks the running process to do a pass. Returns immediately.
Functions
@spec backfill_session_starts() :: non_neg_integer()
Marks the first hit of every session recorded before schema V3, which
introduced session_start — at most 100 batches of
5000 sessions per call, walking session_id in order from a
cursor kept in the web_analytics_session_starts_backfilled setting ("done" once finished).
Returns the number of hits it marked. Sessions recorded since V3 are already marked; meeting one again is harmless (its first hit is the one marked).
Returns a specification to start this module under a supervisor.
See Supervisor.
@spec prune_old_events() :: non_neg_integer()
Deletes raw events past the retention horizon, in batches.
Returns the number of rows deleted. Days past the rollup watermark are left alone — pruning never runs ahead of the aggregation that preserves the trend line — and nothing is deleted when the watermark can't be read.
@spec rolled_through() :: {:ok, Date.t() | nil} | :error
The last day known to be fully rolled up, or nil when nothing has been
rolled up yet. :error when the setting can't be read.
@spec rollup_day(Date.t()) :: :ok | :error
Aggregates one day into DailyStat rows, one per site.
@spec rollup_pending_days(DateTime.t()) :: non_neg_integer()
Rolls up the completed days past the watermark, and re-rolls the last 2.
Returns the number of newly rolled-up days that had traffic. At most 60 days are handled per pass. Stops at the first day whose rollup fails, so the watermark never skips a day.
@spec run() :: %{rolled_up: non_neg_integer(), pruned: non_neg_integer()}
Runs a rollup + prune pass immediately, in the caller's process.
Returns %{rolled_up: days, pruned: rows}. Used by the admin settings page's
"Run now" action and by tests, which need the work to happen on the sandbox
connection.
@spec run_async() :: :ok
Asks the running process to do a pass. Returns immediately.