Replicant.Snapshotter (Replicant v1.5.0)

Copy Markdown View Source

Reads a consistent snapshot of the publication's tables (spec §4) on its own normal Postgrex connection, at the LSN a durable slot exported via EXPORT_SNAPSHOT, and pushes the rows to the sink as %Change{op: :snapshot} batches — then durably sets the sink checkpoint to the snapshot's consistent point (the handoff commit). Spawned and LINKED to Replicant.Connection, which holds the exported snapshot valid by staying idle (proven safe past wal_sender_timeout, spec §11). The link binds the snapshotter's lifetime to the pipeline: a Connection/pipeline teardown mid-snapshot tears the snapshotter down with it, so an orphan can never mutate the sink after the pipeline is gone. Graceful completion exits :normal (which never propagates over the link), so the {:snapshot_done, lsn} / {:snapshot_failed, err} messages still drive the handoff.

Both snapshot modes project only published columns, in physical column order: pgoutput uses pg_publication_tables.attnames on PostgreSQL 15+, pglogical uses set_att_list, and wal2json keeps all configured-table columns. Both readers apply server row filters too: pgoutput ORs INSERT predicates (an unfiltered INSERT publication wins); pglogical ANDs its non-NULL set filters, matching its 2.4.8 output plugin. Conflicting lists halt :snapshot_column_list_conflict before any snapshot row is read or delivered.

Value-free boundary (Critical Rule 1)

A Postgrex query/cursor fault — raised OR returned — can embed row/column values in its message. Every fault is scrubbed to a value-free %Replicant.Error{reason: :snapshot_failed} (only a structural module name kept) before the Connection is told. On success it sends {:snapshot_done, consistent_point}; on any fault {:snapshot_failed, %Error{}}. It reads on a SEPARATE connection, so it never blocks the Connection's keepalive path.

Source-side cost

The consistent-snapshot read holds a single long-running REPEATABLE READ transaction open for the whole backfill (a slow sink extends it), which pins xmin on the source and defers VACUUM there until the snapshot completes.

Summary

Functions

Spawn + LINK the snapshotter to the caller (the Connection) so it is torn down with the pipeline. Returns the pid.

Types

args()

@type args() :: %{
  :snapshot_name => String.t(),
  :consistent_point => Replicant.lsn(),
  :connection => keyword(),
  :publication => [String.t()],
  :sink => module(),
  :reply_to => pid(),
  optional(:mode) => :sink_owned | :lib,
  optional(:tables) => [{String.t(), String.t()}] | nil,
  optional(:replication_sets) => [String.t()] | nil,
  server_version_num: non_neg_integer()
}

Functions

start(args)

@spec start(args()) :: pid()

Spawn + LINK the snapshotter to the caller (the Connection) so it is torn down with the pipeline. Returns the pid.