The order-faithful primary-key core of the initial snapshot (C2, design Ch3/Ch4).
A chunk pages by ORDER BY pk in MySQL while the cursor-gate classifies streamed
changes by k ≤ cursor in Elixir. For that pairing to be correct — no silently
mis-classified change (delivered vs suppressed → gap/dup) — the PK's Elixir term-order
MUST provably match MySQL's ORDER BY. This module enforces that with a positive type
allowlist and supplies the two operations the coordinator needs on PK values:
canonical/2— the equality form for the sink ledger. Canonicalization happens in Elixir (reconciliation-equality), NOT MySQL collation. The text-protocol string form of a PK (a chunk read) and the binlog-decoded integer/binary form (a streamed change) canonicalize to the SAME term, so the ledger reconciles a chunk row against a streamed change to the same key.compare/2— the ordering for the cursor-gate (k ≤ cursor). Over the allowlist it is exactly MySQLORDER BY: integers order numerically,BINARY/VARBINARYorder byte-wise (shorter prefix first), and composites order lexicographically as tuples (matching MySQL row-value ORDER BY).
The positive allowlist (Ch3.2 / Ch4; the string arm is ADR-0012)
ACCEPT the types whose comparison against MySQL ORDER BY is provably faithful:
- signed/unsigned integer —
TINYINT/SMALLINT/MEDIUMINT/INT/BIGINT BINARY/VARBINARY(byte-ordered)CHAR/VARCHAR— collation-ordered, with the server as the only collation oracle: a string PK's canonical form is its collation WEIGHT BYTES (probe-proven: weight-byte order ==ORDER BYforai_ci, both_binweight forms, PAD SPACE over distinct keys, multi-levelas_cs, and composites). The chunk read selectsWEIGHT_STRING(pk)alongside the row; the stream side resolves weights throughresolve_weights/4(a COLLATE-pinnedCONVERT(X'..' USING charset)introducer over the binlog's raw column bytes — an unpinned introducer computes the charset-DEFAULT collation's weights, a different order space). Distinct PK values are collation-distinct by the PK constraint itself, so weights are a faithful key identity.- composites of the above (tuple compare; a string position compares by its weight)
REFUSE everything else with :snapshot_pk_unsupported_type:
- the TEXT family — order semantics are consistent, but a TEXT prefix PK pays a measured
filesort per chunk page (
Using filesortvs VARCHAR'sUsing index;probe/collation_weight_probe.exsQ6b) — superlinear backfill. ENUM/SET— the column'sORDER BYis member-position order and its column weights are position-based, while any stream-side introducer computes STRING weights — two disagreeing order spaces, no uniform mechanism (probe Q13).DECIMAL/DOUBLE/FLOAT/DATE/DATETIME/temporal and any other type — their Elixir term-order diverges from MySQL.
BIGINT UNSIGNED across 2^63 (Ch3.1)
An unsigned 64-bit PK spans [0, 2^64). A signed-wrap decoder reads the high half as
negative (2^63 → -2^63, 2^64-1 → -1), so it would order 2^63… before 0… and the
cursor-gate would mis-classify a high-half key. canonical/2 decodes to the TRUE unsigned
value: a text form parses directly, and a raw integer that arrived signed-wrapped (a
negative for an unsigned column) is normalized back into [0, 2^width).
No PK → fail closed
introspect/2 reads information_schema for the PRIMARY KEY. A table with no PK accepts
a UNIQUE key fallback ONLY IF every column of that key is NOT NULL — a nullable
unique-key column admits NULL-keyed rows that WHERE k > cursor never selects (a silent
gap, Ch3.3), so such a key is refused :snapshot_table_no_primary_key. A table with
neither is refused the same way.
Rule 1
No PK value is ever logged or telemetered here. Errors are value-free atoms. The
information_schema queries embed only the schema/table (structural identity — the same
fields telemetry may carry), never a row value, and are escaped as SQL string literals.
Summary
Types
A canonical PK: a single value (single-column PK) or a tuple (composite PK).
The introspected key: the ordered PK columns, their order-faithful types, and — for the
string columns — the charset/collation pair that pins every weight and cursor literal
(ADR-0012). A non-string column's charset/collation entry is nil (mirroring
information_schema.COLUMNS, where CHARACTER_SET_NAME/COLLATION_NAME are NULL for
non-string columns).
An order-faithful PK column type atom (integer family, :binary, :varbinary, :char, :varchar).
Functions
Builds the canonical equality form of a PK from pk_types and the raw column values
(raw_values, one per PK column in order).
Compares two canonical PK values (from canonical/2): :lt, :eq, or :gt.
Introspects the PRIMARY KEY (or a NOT-NULL-complete UNIQUE key fallback) of
{schema, table} over a Capstan.Query handle.
The canary verdict: the collation-ordered weight sequence equals the weight-ordered one. Ties are inherently safe — collation-equal values carry equal weights, so both sequences show the same bytes at tie positions.
Builds the bootstrap order-contract canary SQL for one string pk column.
The pure key resolution over the two information_schema resultsets — the RED-capable
seam introspect/2 delegates to (so the PK / unique-key fallback / type-refusal logic is
provable without a live server).
Classifies a single column's (DATA_TYPE, COLUMN_TYPE) to an order-faithful pk_type(),
or refuses :snapshot_pk_unsupported_type.
Resolves the collation weight bytes for a batch of raw PK-column values over a
Capstan.Query handle — {:ok, %{raw => weight}}, or a value-free {:error, reason}.
Builds the COLLATE-pinned weight-resolution SQL for a batch of raw column values.
Types
A canonical PK: a single value (single-column PK) or a tuple (composite PK).
@type key() :: %{ pk_columns: [String.t()], pk_types: [pk_type()], pk_charsets: [nil | String.t()], pk_collations: [nil | String.t()] }
The introspected key: the ordered PK columns, their order-faithful types, and — for the
string columns — the charset/collation pair that pins every weight and cursor literal
(ADR-0012). A non-string column's charset/collation entry is nil (mirroring
information_schema.COLUMNS, where CHARACTER_SET_NAME/COLLATION_NAME are NULL for
non-string columns).
@type pk_type() ::
:tinyint
| :tinyint_unsigned
| :smallint
| :smallint_unsigned
| :mediumint
| :mediumint_unsigned
| :int
| :int_unsigned
| :bigint
| :bigint_unsigned
| :binary
| :varbinary
| :char
| :varchar
An order-faithful PK column type atom (integer family, :binary, :varbinary, :char, :varchar).
Functions
@spec canonical([pk_type()], [binary() | integer()]) :: canonical_pk()
Builds the canonical equality form of a PK from pk_types and the raw column values
(raw_values, one per PK column in order).
A raw value is either a text-protocol string (a chunk read) or a binlog-decoded term (a
streamed change); both forms of the SAME PK canonicalize equal. Integers normalize to
their true value (an unsigned column's signed-wrapped negative is unwrapped into
[0, 2^width)); binaries pass through their bytes; a string column's value is the
{raw, weight} pair its caller assembled (the weight being the canonical bytes —
ADR-0012; a bare binary for a string column raises, value-free, rather than silently
comparing raws). A single-column PK returns a bare value; a composite returns a tuple in
column order.
@spec compare(canonical_pk(), canonical_pk()) :: :lt | :eq | :gt
Compares two canonical PK values (from canonical/2): :lt, :eq, or :gt.
Over the allowlist this is exactly MySQL ORDER BY: Erlang term order coincides with MySQL
numeric order for integers, byte-wise order for BINARY/VARBINARY, and — for same-arity
composites — lexicographic tuple order for row-value ORDER BY. Both arguments must be
canonical forms; the caller compares like against like (a table's PK shape is fixed).
@spec introspect( Capstan.Query.t(), {String.t(), String.t()} ) :: {:ok, key()} | {:error, atom()}
Introspects the PRIMARY KEY (or a NOT-NULL-complete UNIQUE key fallback) of
{schema, table} over a Capstan.Query handle.
Returns {:ok, %{pk_columns: [...], pk_types: [...]}} for an order-faithful key, or a
value-free {:error, reason}: :snapshot_pk_unsupported_type when a key column's type is
outside the allowlist, :snapshot_table_no_primary_key when there is no PK and no
NOT-NULL-complete unique key. A transport/query fault propagates its value-free reason
from Capstan.Query.query/2.
The canary verdict: the collation-ordered weight sequence equals the weight-ordered one. Ties are inherently safe — collation-equal values carry equal weights, so both sequences show the same bytes at tie positions.
Builds the bootstrap order-contract canary SQL for one string pk column.
Two queries over the same fixed ASCII vector, introduced with the column's own
charset + collation pin: the first orders by the COLLATION (ORDER BY v), the
second by the WEIGHT BYTES (ORDER BY WEIGHT_STRING(v)), each returning one
hex-encoded weight per row. Under MySQL's documented order contract the two
weight sequences are EQUAL (order_contract_ok?/2) — the gate's every decision
rests on that equality.
The pure key resolution over the two information_schema resultsets — the RED-capable
seam introspect/2 delegates to (so the PK / unique-key fallback / type-refusal logic is
provable without a live server).
index_rows—STATISTICSrows[INDEX_NAME, SEQ_IN_INDEX, COLUMN_NAME, NON_UNIQUE]column_rows—COLUMNSrows[COLUMN_NAME, DATA_TYPE, COLUMN_TYPE, IS_NULLABLE, CHARACTER_SET_NAME, COLLATION_NAME]
(the four STATISTICS cells are NOT NULL; the charset/collation cells are NULL for
non-string columns, so those two cells may be nil). Returns the same
{:ok, key()} | {:error, reason} contract as introspect/2.
@spec resolve_pk_type(binary(), binary()) :: {:ok, pk_type()} | {:error, :snapshot_pk_unsupported_type}
Classifies a single column's (DATA_TYPE, COLUMN_TYPE) to an order-faithful pk_type(),
or refuses :snapshot_pk_unsupported_type.
Signedness rides COLUMN_TYPE (which carries the unsigned attribute); DATA_TYPE alone
cannot distinguish it. CHAR/VARCHAR accept (ADR-0012 — the weight path carries their
ordering); the TEXT family (measured per-page filesort) and ENUM/SET (position-based
order no introducer weight path reproduces) refuse, as does every non-integer, non-binary,
non-char/varchar type (its Elixir order diverges from MySQL).
@spec resolve_weights(Capstan.Query.t(), String.t(), String.t(), [binary()]) :: {:ok, %{required(binary()) => binary()}} | {:error, atom()}
Resolves the collation weight bytes for a batch of raw PK-column values over a
Capstan.Query handle — {:ok, %{raw => weight}}, or a value-free {:error, reason}.
The stream-side arm of ADR-0012: a binlog-decoded string PK arrives as the column's raw
bytes (the row event carries no charset), and its canonical form is the weight the SERVER
computes for those bytes under the column's charset + collation. The batch is chunk-bounded
(128 values per statement) so one bulk-load transaction cannot exceed
max_allowed_packet; a query fault surfaces as the scrubbed Capstan.Query atom for the
caller to budget and halt on.
Builds the COLLATE-pinned weight-resolution SQL for a batch of raw column values.
One WEIGHT_STRING(CONVERT(X'<hex>' USING charset) COLLATE collation) term per raw, in
order — the result row's columns pair positionally with raws. The COLLATE pin is
LOAD-BEARING: without it the CONVERT result carries the charset-DEFAULT collation, whose
weights are a different order space (probed: a column's as_cs multi-level weight vs the
unpinned ai_ci primary-only weight), and the unpinned form in a WHERE additionally
raises ERROR 1267 on non-default collations. Hex literals carry arbitrary bytes (NULs,
quotes, backslashes, empty) with no string-literal escaping surface.