GoodAnalytics.PartitionManager (GoodAnalytics v0.1.1)

Copy Markdown View Source

Manages monthly partition creation for ga_events.

On startup, ensures partitions exist for the current month and 2 months ahead. Uses pgflow when available (preferred), falling back to direct SQL.

Add GoodAnalytics.Flows.CreatePartitions to your PgFlow configuration:

{PgFlow,
 repo: MyApp.Repo,
 flows: [GoodAnalytics.Flows.CreatePartitions]}

The flow provides retry logic, execution history, and optional cron scheduling.

Without pgflow

The PartitionManager will create partitions directly via SQL on startup and every 24 hours.

Recovery from a polluted default partition

When a fresh deploy receives traffic before the manager's first tick, those rows land in ga_events_default because no monthly partition exists for the current period. A naïve CREATE TABLE … PARTITION OF … then raises check_violation because Postgres re-validates the default partition's implicit "no rows belong to a sibling" invariant.

The manager detects this case and recovers without detaching the default partition (which would briefly leave inserts with no routing target):

  1. Acquire a Postgres advisory lock so a single node performs DDL.
  2. Take an ACCESS EXCLUSIVE lock on ga_events.
  3. Stage the offending rows from ga_events_default into an unlogged temp table, then DELETE them from default.
  4. CREATE TABLE … PARTITION OF … (now succeeds because default is clean for that range).
  5. INSERT INTO ga_events SELECT * FROM staged so the rows route into the new monthly child via the parent's partition tree.

All of (2)–(5) run in one transaction. If the manager crashes between (2) and (5) the transaction rolls back; default keeps its rows.

Summary

Types

partition_result()

Per-month outcome from process_partitions/0. :ok means the partition already existed or was created cleanly. :recovered means the partition was created after draining offending rows from ga_events_default. :error means the operation failed and the partition was not created.

Functions

advisory_lock_key()

Returns the advisory lock key for partition management, namespaced by the configured schema prefix.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

create_partition_sql(month_start)

Returns the SQL to create a partition if it doesn't exist.

create_partitions_direct()

Creates partitions directly via SQL (bypasses pgflow). Returns :ok for legacy callers; use process_partitions/0 if you need per-month results.

ensure_initial_partitions()

Pre-creates the initial set of monthly partitions during installation.

ensure_partitions()

Ensures partitions exist, using pgflow if available.

months_ahead()

Returns the configured number of future months pre-created on each tick.

partition_name(date)

Generates the partition table name for a given month.

process_partitions()

Ensures monthly partitions exist for the current month and the next months_ahead/0 months. Returns one partition_result/0 map per month processed, suitable for pgflow run history or operator dashboards.

start_link(opts)

Types

partition_result()

@type partition_result() :: %{
  :partition_name => String.t(),
  :month_start => Date.t(),
  :status => :ok | :recovered | :error,
  optional(:error) => String.t()
}

Per-month outcome from process_partitions/0. :ok means the partition already existed or was created cleanly. :recovered means the partition was created after draining offending rows from ga_events_default. :error means the operation failed and the partition was not created.

Functions

advisory_lock_key()

@spec advisory_lock_key() :: integer()

Returns the advisory lock key for partition management, namespaced by the configured schema prefix.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

create_partition_sql(month_start)

Returns the SQL to create a partition if it doesn't exist.

Bounds are emitted as TIMESTAMP WITH TIME ZONE literals with an explicit +00 offset so the partition range is always interpreted in UTC, regardless of the session timezone.

create_partitions_direct()

@spec create_partitions_direct() :: :ok

Creates partitions directly via SQL (bypasses pgflow). Returns :ok for legacy callers; use process_partitions/0 if you need per-month results.

ensure_initial_partitions()

@spec ensure_initial_partitions() :: [partition_result()]

Pre-creates the initial set of monthly partitions during installation.

Called from the migration generated by mix good_analytics.setup so that current + next-months_ahead/0 partitions exist before traffic arrives. Equivalent to process_partitions/0; named separately for callsite clarity.

ensure_partitions()

Ensures partitions exist, using pgflow if available.

When pgflow is running and the ga_create_partitions flow is registered, starts a flow run. Otherwise falls back to direct SQL creation.

months_ahead()

@spec months_ahead() :: non_neg_integer()

Returns the configured number of future months pre-created on each tick.

partition_name(date)

Generates the partition table name for a given month.

process_partitions()

@spec process_partitions() :: [partition_result()]

Ensures monthly partitions exist for the current month and the next months_ahead/0 months. Returns one partition_result/0 map per month processed, suitable for pgflow run history or operator dashboards.

If a check_violation is raised while creating one of the upcoming months (because traffic landed in ga_events_default for that range before the partition existed), the offending rows are drained into the new monthly child as part of the same call. Broad scans of ga_events_default for older polluted months are NOT performed here — use mix good_analytics.recover_default for that.

Wraps the work in a transaction-scoped Postgres advisory lock so a single node performs DDL across a multi-node deploy. If another node holds the lock, returns an empty list.

start_link(opts)