Updated: 2026-08-13 · Release: v1.1.0 (tagged) · Branch: main
⚠ Commit-SHA note. Git history was rewritten after most of this file was written, so the historical commit SHAs cited in the slice rows below (e.g.
489c6b1,6387e35,a6467af,2699e72) no longer resolve to commits in this repo. They are retained as a historical record of the lifecycle narrative, not as addressable refs. The authoritative current refs are the snapshot below. Test counts in historical rows are milestone evidence and are not claims about the current suite.
Current package snapshot
The initial sequencing plan below is complete, and both packages have continued through later releases:
replicant1.1.0 is published and taggedv1.1.0(a post-1.0 hardening patch — see CHANGELOG[1.1.0]; 1.0.0 shipped atv1.0.0).ash_replicant0.3.0 is published and tagged atc5ef154; a coordinated 1.0 release requiring Replicant~> 1.0is in progress. Its source dependency remains{:replicant, "~> 0.1.0"}until that coordinated release.
Sequencing (user directive, 2026-07-05) — ✅ FULLY EXECUTED
Historical record of the build order, now complete. "One in-build project at a
time": ash_replicant was gated until replicant was "completed in full and
published" = build the remaining §3 slices → publish replicant to Hex → start
ash_replicant. Value-ranked order (1) checkpoint-store → (2) batching → (3)
proto≥2 streaming was executed in full; each slice was closeout-reviewed
(/review-autopilot --fix) to 100/100. With replicant 0.1.0 on Hex the gate
cleared, and ash_replicant has since shipped through to its own 0.3.0 release.
Shipped — v1 streaming core (fail-closed, reviewed, green)
| Slice | What landed | Closeout |
|---|---|---|
| Plan 1 — offline core | decode / assemble / validate / redact; value-free boundary, TOAST sentinel, watermark skip, schema-change classification, identifier allowlist, sink behaviour, telemetry, real-pgoutput byte conformance | 100/100 |
| Plan 2 — live streaming + exactly-once | Connection owns the slot, ack-after-checkpoint, slot-invalidation fail-closed halt, AssemblerServer, per-pipeline supervision, §4 bounded in-flight window, real-PG16 crash-injection (loss=0, effect-dup=0) | 89→100 |
| replicant-snapshot — initial backfill | snapshot: true: EXPORT_SNAPSHOT → COPY at consistent_point → hand off at snapshot LSN, gap-free/dup-free; handle_snapshot/2 + handle_snapshot_complete/1; mid-COPY crash halts :snapshot_incomplete | 100/100 |
Test evidence at that milestone (pre-history-rewrite figures): unit 184/0,
integration 197/0, dialyzer 0. The later 6fe9d3c release milestone recorded
unit 350/0 + integration 38/0 and dialyzer 0; see the current snapshot above
for present package refs.
Delivered — the spec §3 slices (all shipped + closed-out)
Each was a named slice the v1 primitive was fail-closed without — absence
refused partial delivery rather than doing it silently. All are now shipped,
closeout-reviewed to 100/100, and first published in replicant 0.1.0.
| # | Slice | Unlocks | Depends on | Status |
|---|---|---|---|---|
| — | ash_replicant (standalone lib) | The first-class consumer: an Ash/Postgres sink adapter carrying multitenancy + classification, one layer up from the tenant-blind core. This is what turns the primitive into a usable product. | replicant published (gate cleared) | ✅ BUILT + PUBLISHED — 0.3.0 on Hex (tagged at c5ef154; repo /Users/rp/Developer/Base/ash_replicant, current main 4000681). Consumes {:replicant, "~> 0.1.0"} from Hex. Sibling to arcadic→ash_arcadic. |
| 1 | replicant-checkpoint-store | Non-transactional sinks (files, external APIs): a lib-owned checkpoint table with a mandatory checkpoint-after-persist write order (dup, never loss). | Core (shipped) | ✅ Shipped + closeout-ready 2026-07-05 (13/13 tasks, exec-autopilot) · /review-autopilot --fix HEAD 2d23832: graded 89/100 → 100 on user ruling for the sole design-decision; 6 findings fixed, gate battery ALL-PASS 231/0 dialyzer 0. F3 RESOLVED (2026-07-05: build §14.18 next — spawns slice 1b). |
| 1b | replicant-store-fault-retry (§14.18) | Bounded-retry-then-halt on a checkpoint-store fault: a transient blip self-heals within N; a persistent outage retries N times then HALTS + alerts the operator (replaces the current UNPACED connect-retry interim + the immediate mid-stream write halt). Closes checkpoint-store closeout F3. | replicant-checkpoint-store (shipped) | ✅ Shipped 2026-07-05 (exec-autopilot; 7/7 tasks, per-task two-stage opus review; unit 224/0, integration 25/0, dialyzer 0, credo/format clean; 0 tier escalations, 3 test-hardening review-fix rounds T1/T5/T6). Closeout /review-autopilot pending. Design adversarially reviewed (9 challenges 8-acc/1-refuted); plan machine-gated + independently reviewed (5/5 fixed). |
| 2 | replicant-batching | Throughput: batched checkpointing for lib mode — defer the lib-owned checkpoint write + slot ack to once per batch of N txns (sink delivery stays per-txn). Amortizes the synchronous serial store round-trip. Per-transaction checkpointing is the correctness baseline it optimizes. | Core + checkpoint-store + store-fault-retry (shipped) | ✅ Shipped + closed-out 2026-07-06 (exec-autopilot 5 tasks 2 opus/3 sonnet, 0 tier escalations/0 review-fix rounds → /review-autopilot --fix 100/100 grader-verified; post-fix HEAD 7bfa193, unit 249/0 + integration 28/0, dialyzer 0, credo 677/0, format/compile clean). Commits: config c22f03e, assembler/server 2ef4d7d, pipeline 3c4734b, integration ce9bf25, docs 904d153; closeout fixes 113a2a4/65040f3/7a9812f/c6c1c48/85672f1/7bfa193. Both closeout design-decisions user-ratified + implemented (spec §15 amendments): LSN-span base → max(lib_checkpoint, stream_floor); batch discarded on mid-stream reconnect. Cross-vendor Codex: 2 uniques (reconnect-stale-batch, value-free-leak). Opt-in checkpoint_store: [batch: [max_transactions: 100, max_delay_ms: 1000]] + auto LSN-span lag-cap (max_inflight_lag/4); dup bound widens to one batch (crash + graceful stop + mid-stream reconnect), loss=0 unconditional. Sink-owned batching → replicant-batch-delivery ✅ executed 2026-07-06 (row 2b). |
| 2b | replicant-batch-delivery | Sink-owned (transactional) batching: a new handle_batch sink callback delivering N transactions as ONE atomic unit — amortizes the sink's own commit cost (distinct from replicant-batching's library-owned checkpoint-write amortization). Carved out of replicant-batching because it needs a mechanically distinct delivery contract + atomic-multi-txn semantics (a fresh-context reviewer confirmed the boundary is mechanical, not effort). | replicant-batching (shipped) | ✅ Executed 2026-07-06 (exec-autopilot, 9 tasks: 5 opus / 3 sonnet / 1 haiku; per-task two-stage opus review, 4 review-fix rounds total, 0 tier escalations). Post-sweep HEAD 6387e35 (range 7bfa193..6387e35, 8 code/test/docs commits): unit 271/0 + integration 31/0, dialyzer 0, credo 741/0, format/compile clean (gate-log 20260706-071040-bd-full-sweep-6387e350f1). Effect-once (dup=0) crash-injection marquee GREEN against live PG16 (mid-batch atomic-checkpoint fault rolls the whole handle_batch txn back → resume re-delivers dup=0). Designed via brainstorm-autopilot, user-approved; fresh-context adversarial review folded — 11 challenges / 3 blocking, all reconciled. Effect-once preserved (dup=0, loss=0) — the transactional idempotent sink writes N txns + checkpoint atomically; stronger than batching's dup≤batch. Opt-in top-level batch_delivery: [max_transactions: 100, max_delay_ms: 1000] (sink-owned only; + checkpoint_store ⇒ :config_invalid; missing handle_batch/1 ⇒ :batch_unsupported). 6 lib modules (sink/config/assembler/assembler_server/pipeline/connection); telemetry unchanged. CLOSED-OUT 2026-07-06 via /review-autopilot (8 lenses): 97 → 100 after one in-session fix. The lone should-fix (flush_sink_batch's result case lacked a value-free catch-all → a non-conforming handle_batch RETURN raised CaseClauseError in the flush path — which runs via do_flush, outside handle_message/2's value-free rescue — leaking the returned term into the OTP crash log, Critical Rule 1; caught by the security lens and the Codex cross-vendor lens, duplicate/live-repro'd) was fixed in commit 215d58b (value-free _unexpected catch-all + red-capable test RED→GREEN; spec §8 matrix + §14.19 amended). Fresh full battery @ 215d58b: credo 744/0, dialyzer 0, unit 303/0, integration 31/0 (marquee dup=0 green). |
| 3 | replicant-streaming | pgoutput proto ≥ 2 in-progress-transaction streaming (opt-in streaming:): decode Stream Start/Stop/Commit/Abort + (sub)xid-prefixed changes, reassemble each streamed txn in memory demultiplexed by top-level xid (interleave + subtransaction-abort filtering), deliver the complete %Transaction{} on Stream Commit through the unchanged sink contract. Primary benefit is server-side (walsender streams a large txn instead of buffering it whole); consumer memory bounded by the existing §4 halt. | Core (shipped) | ✅ Executed 2026-07-06 (exec-autopilot, 10 tasks 7 opus/3 sonnet; per-task two-stage opus review, 0 tier escalations). Post-sweep HEAD a6467af (range 2901d4f..a6467af, 17 code/test commits): unit 307/0 + integration 34/0 (baseline 272/31 → +35 unit / +3 marquee tests, 0 regressions; the plan's recorded "303" unit baseline was a mis-record — true pre-slice count 272, per the batch-delivery closeout), dialyzer 0, credo 814/0, format/compile clean (gate-logs 20260706-1610…1612, HEAD-bound). Streaming effect-once (dup=0) crash-injection marquee GREEN on live PG16 — a large streamed txn with a nested-savepoint rollback (ids 1001..1500) delivers effect-once with the sub-aborted range filtered ABSENT end-to-end; streaming proven fired ([:replicant, :stream, :committed] probe, RED when the work_mem forcing is removed). Real spec-§7 defect found + fixed by the marquee + an adversarial investigation: StreamCommit delivered an EMPTY streamed txn (a spilled unpublished-only txn streams under pgoutput v2 but is skipped by v1), violating "a streamed txn is indistinguishable from a non-streamed one at the sink" — fixed via {:skipped} empty-suppression (fe01f0c); the masked-assertion workaround was caught and reverted to the honest calls_count == 1 live red-gate. Opt-in top-level streaming: [max_concurrent_txns: N] (default 64); Config streaming: nil ≡ disabled (parity with batch_delivery). Sink contract + telemetry.ex UNCHANGED; effect-once preserved (dedup pre-skip + subxid-abort filter). Designed via brainstorm→plan-autopilot (design fresh-context-reviewed — 9 challenges/3 blocking folded; plan machine-gated 0-errors + independently reviewed 5/5). Closeout /review-autopilot --fix 2026-07-06 → 100/100 (fresh-context grader), post-fix HEAD b9e0947 (range 11f6afc..b9e0947). 8 lenses + adversarial refuter + worktree-isolated fix-diff re-reviewer. 1 blocking cross-vendor UNIQUE catch FIXED: an EMPTY StreamCommit returned {:skipped, lsn} which acks the slot immediately — because streaming composes with the batch modes (§7) and a buffered-but-unflushed txn's commit_lsn is BELOW the empty txn's lsn (commit order), the ack advanced confirmed_flush PAST un-delivered/un-checkpointed WAL = loss=0 violation (sink-owned batch_delivery); fixed by folding the empty txn's lsn into the open batch (suppress_empty_stream_commit/2, 123287e; red-capable test mutation-proven; same-family lenses reasoned stream-interleave but missed the batch composition). Cross-vendor CV2 (streamed Relation cache scoping) adversarially REFUTED (§8-conformant halt + Enum.zip truncation + pgoutput Relation re-send). 2 should-fixes fixed: §8 halt reason reconciled to :too_many_streams (123287e), byte-level xid-tamper decoder test §12.2 added (75c0947); replay collapsed to one-pass map_reduce. Post-fix full battery GREEN on live PG16 @ b9e0947 (clean tree, gate-run --verify): unit+integration 344/0, integration marquee 34/0, dialyzer 0, credo 815/0, compile/format clean (baseline 341→344, +3 unit tests, 0 regressions). See spec §15 closeout amendments. |
| 3-spill | replicant-streaming-spill | Consumer-side disk spill so a single transaction larger than memory streams through without halting — the "unbounded single-txn size" capability. Carved from row 3 (its first-disk-I/O + PII-at-rest + OTP-lifetime risk surface earns its own review); row 3 is fail-closed without it (a too-large streamed txn halts, as under v1). | replicant-streaming | ✅ Executed 2026-07-07 (exec-autopilot 12 tasks [9 opus / 3 sonnet] — 5 review-fix rounds, 0 tier escalations; unit 383/0 + integration 38/0 live PG16, dialyzer 0, credo 903/0, format/compile clean; range 9d9821e..473cef9; gate-log 20260707-042549-spill-full-sweep-473cef984a). Designed via brainstorm-autopilot Stage-6 fresh-context adversarial review (9 challenges / 4 blocking reworked). Delivery contract user-ratified — a >RAM txn delivers as a lazy disk-backed %Transaction.changes (Enumerable.t()); the parent's event-based "incremental streaming sink contract" non-goal is OBVIATED (not deferred) — lazy whole-Transaction changes gives the same memory capability while preserving atomicity. Two ceilings: resident RAM max_inflight_lag (spill trigger) + disk max_spill_bytes (:spill_exhausted halt); §4 numerator now received − floor − spilled (the change the parent deferred here). One Replicant.Spill module = the sole File.* + at-rest boundary (0700 dir / 0600 files, global startup sweep, value-free :spill_io_failed); ephemeral non-fsync'd scratch (write-once-read-once-within-one-lifetime); no in-lib encryption (key-mgmt ownership, NOT Rule 5 — a clean future). Plan-autopilot: 12 tasks (9 opus / 3 sonnet), machine-gated (plan-verify 0/0) + fresh-context reviewed (7 findings / 7 fixed — 2 blocking: Error.reason() enum gap → dialyzer fail; lib+batch spilled-file happy-path leak). Exec DONE — oversized-txn spill effect-once (dup=0) marquee GREEN on live PG16 (append-only ledger, [:stream,:spilled] probe fired), no stale spill files, both ceilings halt (:spill_exhausted disk + §4 RAM). The crash-injection marquee CAUGHT A REAL §8 BUG — a per-txn spilled delivery orphaned its spill file on a sink-fault halt (deliver_now deleted only on {:ok}) → fixed dab7f2f; a live-probe quality review also caught a §4-halt evasion (stale spilled_bytes mirror survived reconnect via handle_connect, init/1 runs once) → fixed in 191284f. CLOSED-OUT 2026-07-08 via /review-autopilot --fix (fresh-account resume): all 6 findings FIXED (TDD red→green), post-fix HEAD 489c6b1; full battery GREEN (gate-log 20260708-064419-spill-closeout-fix2, nonce c40733be685617fe): unit 387/0 + integration 38/0 live PG16, dialyzer 0, credo/format/compile clean. Fixes: CV1 sink_batch_failed deletes migrated spill files on the FAULT branch — no cleartext orphan (9e57f11); empty-suppression closes the spill FD via Spill.discard (not Spill.rm) — no :emfile leak (97edc74); CV2 append-before-account reorder so a single row > max_inflight_lag spills, {:spilled_bytes} cast now sends the NET post-dispatch total (3ad3cbc); decode_frame validates the frame shape at the at-rest boundary → value-free :spill_io_failed (5561b23); :spill_exhausted telemetry emitted at the disk-ceiling breach (489c6b1). CV3 (§4 ceiling — this spec's own contradiction) resolved via Option A: the CODE is correct (received−floor−spilled vs the SUM max_inflight_lag+max_spill_bytes); §1/§4/§8 text wrongly said max_inflight_lag and was amended. A "Design 2" attempt to CHANGE the code (drop −spilled) was CAUGHT by the live disk-ceiling marquee (it made §4 pre-empt the disk ceiling → :spill_exhausted unreachable) and reverted — the −spilled numerator is LOAD-BEARING (adaptive: routes oversized single txns to the disk ceiling, non-spilling backlogs to §4).** Also de-vacuumed the "disk ceiling" marquee (it always halted via §4 :sink_too_slow — the disk-ceiling halt is deferred to StreamCommit, which a never-committing oversized txn never reaches — now asserts the real reason). See [[replicant-streaming-spill-slice]], [[replicant-fault-path-resource-cleanup]]. Spec closeout amendment added. |
| — | Multi-sink fan-out per slot | Deliberate non-goal, no slice: one slot = one sink = one checkpoint keeps the watermark tractable. Compose a dispatching sink or run multiple slots. | — | Won't build |
Backlog — enhancement opportunities (2026-07-09 code-probe audit)
Complete inventory from a code-level capability audit at HEAD 499c8ba (all
claims probe-verified against lib/, not doc-skimmed). Three sources: (A) new
gaps found by the probe, (B) future slices already named with owners in the
shipped specs' §3 non-goal tables, (C) ecosystem/adoption-layer candidates
(new sibling libs, like ash_replicant). Value-ranked build order:
B1 → A1 → A5 → A2+A3 (paired) → remainder.
A. Code-probe gaps (not previously in any spec)
| # | Item | Unlocks | Evidence | Status |
|---|---|---|---|---|
| A1 | Idle-slot heartbeat / ack-advance | Prevents indefinite WAL pinning on a quiet publication in a busy cluster — the #1 real-world logical-replication incident class. Ack wal_end on keepalive when zero txns are in flight (safe: filtered WAL carries nothing for the publication); today every keepalive acks the checkpointed LSN unconditionally. Touches the exactly-once ack seam → full brainstorm→spec rigor despite small size. | connection.ex:6-9 ("never the received wal_end"); README delegates WAL-lag to ops monitoring | ✅ CLOSED-OUT 2026-07-11 = 100/100 (/review-autopilot --fix, 8 lenses + cross-vendor Codex + fresh-context grader; grader-concurred zero-open). Full lifecycle brainstorm→spec→plan→exec→review, 9 commits abcf22f..34a208d; final battery VERIFIED green: 518 tests/0 failures live PG16, dialyzer 0, credo 1266/0, compile --force --warnings-as-errors + format clean, tree clean; gate-log 20260711-041642-idle-ack-closeout-fix-34a208d0de. Idle predicate not in_txn and open_streams empty and checkpoint_lsn ≥ last_commit_lsn (transaction-boundary; the approved received_lsn ≤ checkpoint_lsn was CAUGHT broken at plan-time — 48-byte tail gap). Closeout fixed 2 should-fix + spec-doc: CV1/CV2 (9dd1ec5) — the single in_txn boolean under-counted proto-v2 CONCURRENT + savepoint streamed txns → open_streams xid-keyed MapSet (StreamAbort closes only on whole-txn abort); CV1 was a CROSS-VENDOR (Codex) UNIQUE catch, missed by all 5 same-family lenses (not constructible loss — §3.2 commit-boundary backstop — but the primary invariant was wrong). A2 (34a208d) idle-advance telemetry tagged kind: :idle. Every task RED-before-GREEN; 3 live marquees each tamper-proven non-vacuous; 0 tier escalations. Standby marquee UNRUN (no substrate) — named coverage gap (spec §3.5). |
| A2 | Logical decoding Message ('M') decode | Outbox-pattern + heartbeat-table consumers (pg_logical_emit_message). Decoder has no Message struct (frames fall to Unsupported); START_REPLICATION omits messages 'true'. Composes with A1. | messages.ex (14 types, no M); query_builder.ex:40-47 | 🏗️ BUILT 2026-07-12 (exec-autopilot; 10 of 12 tasks executed — Task 11 now CLOSED at closeout, see below; range b75f202..e7682b3, 11 commits). Gate battery VERIFIED green at HEAD e7682b3 (clean tree): 519 tests/0 failures (66 excluded integration — no live PG), dialyzer 0, credo 1386/0, compile --force --warnings-as-errors + format clean. Opt-in messages: true; a sink missing handle_message/2 is rejected at START (:messages_unsupported, never silently dropping later). Guarantees (Critical Rule 3 honesty): transactional messages (flags=1) ride %Transaction{messages: [...]} and inherit the txn path's commit_lsn effect-once dedup; non-transactional messages (flags=0) route to handle_message/2 and are at-least-once — no dedup key, duplicates possible on reconnect (stated in the handle_message/2 docstring). Two durability seams prevent silent loss: idle-ack track_txn bump (§8.1 — a non-txn message in flight blocks the idle advance, so keepalive can't advance confirmed_flush past an undelivered message) and the batch-boundary {:flush_before_message} seam (§8.4 — a non-txn message flushes an open sink-owned batch in delivery order). %Message{} struct (transactional?, lsn, prefix, content, xid, ordinal) decoded by v1 + streamed clauses; messages flag threads to start_replication. ⏸ Task 11 DEFERRED (non-blocking handoff): the real-byte 'M' conformance-capture fixtures (corroboration layer in conformance_test.exs) were NOT captured — three capture approaches failed (pg_logical_slot_peek_changes returns text mangling non-printables; pg_recvlogical across docker exec = 0-byte files; a postgrex ReplicationConnection capture timed out without :xlogdata frames). The decoder IS conformance-validated (Task 5's hand-crafted bytes prove the parser; the existing real-byte fixtures corroborate the shared decode path) — the independent 'M'-specific real-byte second layer was missing. ✅ Task 11 now CLOSED (2026-07-14 closeout): real 'M' bytes (non-txn flags=0 + txn flags=1) captured from live PG16.14 via a Postgrex.ReplicationConnection and added to conformance_test.exs (tamper-proven) — the ReplicationConnection capture works; the prior "timed out / uncapturable" claim was false on probe. ✅ CLOSED-OUT 2026-07-14 (forge T2 closeout, live PG16.14 @:5599; commit 5b3680f) — fresh-context spec-conformance + security + gate-integrity lenses + cross-vendor Codex. Caught + fixed a REAL silent-loss bug (spec §7.1): a proto-v2 streamed txn carrying only a transactional message with zero published row-changes was empty-suppressed → the message was DROPPED (v1 delivers it). Fix in assembler.ex deliver_or_skip_stream (suppress only when no surviving messages), red-first regression test in assembler_test.exs; cross-vendor (Codex) corroborated the same bug independently. Security lens: Critical Rules 1/2/3 empirically CLEAN (malformed 'M' → :decode_failure no byte leak; SQL-injection attempts rejected; guarantee-honesty docstring accurate). The integration marquees (messages_test.exs 7/7, multipub_test.exs 3/3) were driven to non-vacuity for the FIRST time — 3 never-run-marquee test-harness bugs fixed (row-shape, hand-rolled cp-table missing updated_at, hard-coded commit_lsn=1). Final battery mix test 586/0 live PG16 (2 :pg17 excluded), dialyzer 0, credo 1388/0, compile -Werror + format clean. ADR-0001. 2 advisory follow-ups ✅ COMPLETED 2026-07-14 in 7df3b69 (T1, fresh-context reviewed clean): streamed-message ordinal now shares the per-txn counter with changes (interleaves correctly, in-memory + spilled — mirrors v1); [:message,:received] telemetry now fires for txn messages (transactional: true) across all delivery modes. Record correction: HEAD 71594ea's (A2+A3 closeout) subject was docs-only; the closeout review actually ran in 5b3680f. |
| A3 | Multi-publication per pipeline | publication_names accepts a comma-list per pgoutput; config takes exactly one :publication. Small config-surface change, real capability. | config.ex:51; query_builder.ex:47 | 🏗️ BUILT 2026-07-12 = COMPLETE (exec-autopilot; 4 of 4 tasks, range b75f202..9c13b2b). publication: String | [String] — a single string is the byte-unchanged default path; a list enables multi-publication, each name validated via Identifier.validate/1 and normalized to a list. start_replication/3 + the four discovery queries (publication_tables/1, pk_columns/0, table_columns/0) bind DISTINCT ... pubname = ANY($1); publication_exists/1 interpolates a validated IN (...) list (the connect-chain simple-query protocol can't bind $1). A NEW connect-chain :publication_check step fails closed if the found-pubnames set ≠ the requested set (decision #18 — START_REPLICATION with a missing pub silently streams the subset). pgoutput dedups overlapping tables across publications (probe-verified). In the same gate battery as A2. ✅ CLOSED-OUT 2026-07-14 (forge T2 closeout with A2, live PG16.14 @:5599; commit 5b3680f) — fresh-context spec-conformance + security + gate-integrity + cross-vendor Codex, all CLEAN on the A3 surface. Probe-verified live: DISTINCT discovery dedup (a table in two publications → pk_raw = {id}, not {id,id}) and the fail-closed :publication_check (found set ≠ requested → halt_publication_missing, stay-idle, value-free). The multipub snapshot-discovery marquee was fixed (the sink lacked handle_snapshot/2 + handle_snapshot_complete/1) and now runs a REAL snapshot proving the DISTINCT union delivers the shared table exactly once. ADR-0002. Same battery: mix test 586/0 live PG16. |
| A4 | origin filter option | Bidirectional / blue-green replication setups (pgoutput PG16+ origin 'none'). Absent from the options builder. | query_builder.ex:40-47 | Open |
| A5 | PG17+ forward-compat + failover slots | Invalidation detection is PG16-keyed (wal_status/conflicting); PG17 adds invalidation_reason (+ failover/synced). PG17 failover = true slots → HA resume-on-promoted-standby. Correction: "CI matrix is PG16-only" was false — there was NO CI; added in this slice. | query_builder.ex:123-127 (comments acknowledge the PG17 divergence) | 📐 DESIGNED 2026-07-11 (brainstorm-autopilot; probe-grounded on live PG16.14 + disposable PG17.10). Fresh-context reviewed — 7 challenges folded: grammar reversed to dual/byte-safe (published default path untouched, FAILOVER only on the opt-in parenthesized path), PG16-failover stay-idle halt (mirrors halt_store_permanent), synced-unpromoted fail-closed halt (livelock→halt), invalidation_reason as authoritative fail-closed signal (fixed atom map, Rule-1 clean, forward-robust), executed dual-version DoD + GitHub Actions PG16+PG17 matrix. 🏗️ BUILT 2026-07-11 (exec-autopilot; 11 tasks [8 opus / 3 sonnet] + hotfix A1, per-task two-stage opus review, 3 review-fix rounds, 0 tier escalations). Gate battery VERIFIED green on BOTH substrates (HEAD 24c5670, tree clean): unit 484/0, PG16 integration (path-scoped, isolated) 52/0, PG17 integration (isolated) 56/0 (pg17 tests EXECUTED, not skipped), compile --force --warnings-as-errors + format clean, credo 1291/0, dialyzer 0; gate-logs 20260711-09*-a5-*. Task 8's PG16 integration run CAUGHT a real regression — replication-connection results arrive as TEXT ("160014", "t"/"f"), so the version gate ("160014" >= 170000 = true) forced the PG17 4-col query on PG16 → undefined_column → reconnect storm on every PG16 pipeline; hotfix A1 (76ffc8d) coerces version/in_recovery/conflicting/synced at the connect boundary (also fixed a pre-existing latent conflicting-text bug). Dual/byte-safe grammar (published default path byte-unchanged), PG16-failover stay-idle halt, synced-unpromoted halt, invalidation_reason authoritative signal, GitHub Actions PG16+PG17 matrix (fails on unready substrate). Commits 3a5ff8d..24c5670 + closeout fixes af6d8fd, 520906a. ✅ CLOSED-OUT 2026-07-11 = 100/100 (/review-autopilot --fix, fresh-context grader-concurred, zero open) — 6 read-only lenses + worktree-isolated tamper pass (all 5 safety-property tests RED-capable, zero vacuous) + independent grader. 2 should-fix fixed: test_helper server_version_num/0 now fails closed when REPLICANT_TEST_URL is set-but-unreachable (was fail-open → vacuous-green risk) af6d8fd; spec amended with the A1 text-coercion mechanism + §2/§5.1 corrections. 1 advisory fixed: orphaned is_in_recovery/0 re-homed to incremental.ex standby?/1 (de-dup + de-orphan) 520906a. Final battery green on BOTH substrates (HEAD 520906a, tree clean): PG16 538/0, PG17 540/0 (pg17 EXECUTED, 0 excluded), credo 1291/0, dialyzer 0; gate-log 20260711-102409-a5-closeout-final. Cross-vendor SKIPPED (Codex rate-limited, 0 probes). The 1st battery's 2 streaming_spill timeouts were load-induced flaky (untouched subsystem; isolated re-run 4/0, quiet-machine battery 0 failures) → backlog-routed to widen the marquee timeout. |
| A6 | Replication-command-error fail-closed policy | A persistent %Postgrex.Error{} from a replication command currently {:disconnect, :query_error} → auto_reconnect livelocks (no halt, no surfaced cause). Bounded-retry-then-halt like §14.18 store-fault-retry, without inspecting the value-free error (Rule 1). | connection.ex handle_result(%Postgrex.Error{}) | ✅ CLOSED-OUT 2026-07-14 (forge T2, live PG16.14 @:5599) — a chain-progress watchdog counts established-then-dropped connect cycles in handle_disconnect (store-retry exempt), enforces max_command_retries (default 5, 0=halt-now) in handle_connect before any store read via the shared CheckpointStore.retry_decision/2, resets on the first replication frame (?w/?k), and stay-idle halts emitting value-free [:connection, :command_error_halt]. Bounds the WHOLE pre-frame class (create-slot, START_REPLICATION, :unexpected_result), not just handle_result — server-down + post-frame transients still self-heal. Design adversarial pass caught+fixed a broken first mechanism (counter-through-{:disconnect} never persists); plan review + per-task review + live marquee (halt at attempt=budget+1, budgets 2→3 & 5→6). Range 9ca211a+981252e (+ core 23ecf47..b14c760); local Forge design and plan receipts retained under .forge/. |
B. Spec-named future slices (owners assigned in shipped specs)
| # | Item | Unlocks | Named in | Status |
|---|---|---|---|---|
| B1 | replicant-snapshot-incremental — resumable / incremental snapshot (DBLog-derived chunking adapted to READ-ONLY sources, interleaved with the stream) | Large-table onboarding: today a crash at 99% of a multi-hour COPY redoes everything (all-or-nothing per attempt, exported snapshot dies with its session). The biggest adoption blocker. | snapshot spec §3 | ✅ CLOSED-OUT 2026-07-10 (/review-autopilot --fix, 8 lenses + cross-vendor Codex + fresh-context grader) = 100/100 (excellent) — 3 fix rounds (69→83→100) + Tasks 12–13 built + all findings closed. HEAD 3ecee0a, battery VERIFIED green (unit+integration 497/0 live PG16, dialyzer 0, credo 1242/0, format/compile clean, tree clean; gate-log 20260710-205502-inc-t12t13-final-3ecee0a368). ALL THREE BLOCKING data-loss defects the review found are FIXED: (1) DELETE/TRUNCATE during backfill crashed the AssemblerServer via pk_tuple(nil, …) OUTSIDE the value-free rescue (OTP row-value leak = Rule 1 breach) + ghost rows → 545fb06 (PK from old_record; truncate→taint); (2) cross-vendor UNIQUE — PK-changing UPDATE resurrected the old key → 9f54f06 (track BOTH keys); (3) CV2 — a keyed table's buffered final chunk discarded after the reader marked it done = loss → b795650 (keyed final-chunk completion barrier). Earlier: 8c8c7db/196140c/b67ee6c (keyless data-loss fixes) + a9a6e1f (lib+batch PK-retention drop-filter, LIFTED the interim lib-batch×incremental rejection) + 4eca198 (§9 chunkretried / compressed-token DoS guard / bounded last_buffered_changes). F7 RESOLVED (dd8f920, user-ratified): resume no longer backfills a mid-run-added table (matches §3/§18 non-goal). TASKS 12–13 BUILT + all findings closed: §12 live-marquee matrix filled — PK-less contended (:snapshot_table_contended+retry), completion at-least-once (lib-mode resume + durable complete?), batch_delivery×incremental effect-once (append-only chunk-PK ledger), :sink_too_slow §4 backpressure DURING backfill — all non-vacuous / red-capable on live PG16 (43eb452, 3ecee0a); feature docs added to README + Replicant moduledoc + usage-rules (7c6c064). Notes closed: post-completion window-clear + real :completed duration + close_table dead-code removed (74f2ccc); cross-run test isolation (drop `cp/prog_/default-progress row on_exit) + resume-marquee timeout +PG16.wait_untildoc (3ecee0a). RI-USING-INDEX-non-PK delete edge routed B2-adjacent (PK-keyed drop-set correct under default RI=PK/FULL; diagnosis on-disk). Fresh-context grader **concurred 100/100 zero-open**; spec closeout amendment + 8-lens reports + 25-finding table on-disk (gitignoredsuperpowers/`). |
| A6b | SQLSTATE-class fast-halt for command errors | Halt IMMEDIATELY on an unfixable replication-command error class (insufficient_privilege / syntax_error / undefined_object) instead of burning the full max_command_retries budget — a halt-latency optimization on the one channel (handle_result(%Postgrex.Error{})) that carries a SQLSTATE. Rule-1-conformant via a SQLSTATE→fixed-atom map (the landed invalidation_reason_atom/1 precedent), so deferred on scope, not prohibition. | A6 design §Non-goals + A5 spec §11 | Open — named by A6 (2026-07-14); the count-based A6 watchdog already bounds every channel correctly |
| B2 | Filtered / column-projected snapshot | Subset snapshots beyond the publication's natural scope | snapshot spec §3 | Open |
| B3 | Parallel multi-table COPY | Snapshot throughput (v1 is serial) | snapshot spec §3 | Open |
| B4 | Proto v3 two-phase streaming (Stream Prepare) | 2PC-aware pipelines | streaming spec §3 | Open |
| B5 | Proto v4 parallel streaming | Concurrent apply of interleaved streams | streaming spec §3 | Open |
| B6 | At-rest spill encryption | In-lib blanket encryption of spill files ("a clean, unblocked future addition"); key management is the separable subsystem | spill spec §3 | Open |
| B7 | Spill-file compression | Disk/CPU trade-off under the spill format | spill spec §3 | Open |
| B8 | Checkpoint-read cache | Eliminates the per-Commit live sink.checkpoint() read; deferred pending profiling evidence that the read matters | batch-delivery spec §3 | Open — profiling-gated |
| B9 | Snapshot column metadata via context | Full column metadata at snapshot time (values arrive natively typed today, so none is needed to interpret them) | snapshot spec §138 | Open |
C. Ecosystem / adoption layer (sibling-lib candidates)
| # | Item | Unlocks | Status |
|---|---|---|---|
| C1 | A proposed metrics/0 helper on Replicant.Telemetry (Telemetry.Metrics definitions) | Oban/Broadway convention — 27 structure-only events exist; every consumer currently hand-authors metric definitions | Open |
| C2 | Broadway producer adapter | Opens replicant to the dominant Elixir data-pipeline ecosystem | Open |
Closed boundaries (cited, not silently dropped)
| Item | Ruling |
|---|---|
| Multi-sink fan-out per slot | Won't-build, user-ratified (core spec §3; row above) — compose a dispatching sink or run multiple slots |
| fsync'd / durable spill | Never, by design (spill spec §3) — spill is scratch, not a WAL; crashes re-stream |
| Incremental streaming sink contract | Obviated (spill spec §3) — lazy whole-Transaction changes delivers the same memory capability with atomicity preserved |
Why ash_replicant was the #1 value item (retrospective)
Value-ranking rationale, retained now that it has shipped as 0.3.0:
- It is the reason the core exists. README states it outright: replicant is
"the reliable CDC consumer sibling to
arcadic," with multitenancy, classification, and Ash resources living "one layer up, in theash_replicantsink adapter." Without it the core had no first-class consumer —ash_replicant0.3.0 is now that consumer. - The other slices optimized a primitive that already worked. Checkpoint-store, batching, and proto-v2 each added value only under a specific pressure (non-transactional sinks / throughput / unbounded single-txn size); none delivered new end-user capability the way the consumer layer does.
- Sequencing note. Despite the #1 value ranking, the user directive built the
§3 slices first (one in-build project at a time) and gated
ash_replicanton thereplicantpublish — which is how it actually shipped.
1.0.0 Release hardening — readiness assessment (2026-08-12)
A four-lens assessment (architecture, security, coverage, API-stability) of the 0.3.1 core against a 1.0.0 bar. The core is release-ready — no P0 in library behavior. The rows below are the gaps to close at/around the 1.0 tag. Status here is authored narrative (this section pre-dates forge-roadmap schema migration); the slug is the slice join key.
| ID | What | Acceptance | Depends | Why |
|---|---|---|---|---|
| D1 | Stale install constraint — README + getting-started Livebook ship {:replicant, "~> 0.2"}, which resolves < 0.3.0 and locks users out of every 0.3 feature (and 1.0). slug:d1-install-constraint | README.md + notebooks/getting_started.livemd show ~> 1.0; grep finds no ~> 0.2 / ~> 0.1.0 install reference | — | Currency — the most-copied snippet |
| D2 | Coordinate the Replicant/AshReplicant major contract — Replicant 1.0 must expose authoritative actual-session identity before checkpoint lookup, and AshReplicant must not admit a 0.3 install that lacks it. slug:d2-ash-replicant-coord | Replicant's fetched package exposes SessionIdentity and handle_session_identity/2, the live callback reports system/database identity from the exact replication connection before checkpoint lookup, AshReplicant requires Replicant ~> 1.0, and the two release in dependency order without moving or deleting a tag | D1 | Release coordination — a 1.0 that orphans its consumer or permits an unsafe downgrade |
| D3 | Release hygiene — toolchain/CI skew, a previously red format gate, an unenforced audit alias, and a vulnerable Postgrex floor cannot ship in 1.0. slug:d3-release-hygiene | .tool-versions pins Elixir 1.20.3-otp-29 / Erlang 29.0.3; CI matches it; format, audits, and gate red-probes pass; Postgrex is at least 0.22.4; actions and database images use immutable revisions | — | A red gate and an unpatched dependency cannot ship 1.0 |
| D4 | :batch type trapdoor — Replicant.Config.t lists optional(:batch) but fetch_batch/3 rejects a top-level :batch with :config_invalid (it is derived). slug:d4-batch-type-trapdoor | The public type no longer advertises a key the user cannot set; dialyzer + compile clean | — | Freeze the type honestly |
| D5 | v1 snapshot casting divergence — snapshot: true runs SELECT * and zips raw Postgrex-decoded values (no cast), so a timestamp arrives as NaiveDateTime from the snapshot and DateTime from the stream; the incremental path was fixed (::text + cast_record) but v1 was never back-ported. slug:d5-snapshot-casting | A red-first convergence test proves a typed column delivers the SAME runtime type from v1 snapshot and the stream; v1 routes values through the shared Casting.Types.cast_record/2 (or v1 is explicitly deprecated); value-free boundary intact | — | Convergence correctness (Critical Rule 1 boundary preserved) |
| D6 | %Transaction.changes spill hazard — typed Enumerable.t(), but a spilled txn is single-pass / call-lifetime; a sink authored against List breaks on its first oversized spilled txn. slug:d6-changes-spill-type | The type names both forms as either [Change.t()] or Spill.Reader.t(), and a @typedoc states the single-pass contract; dialyzer clean | — | The biggest latent data-contract hazard |
| D7 | Hex description overclaims exactly-once — mix.exs description ends "…exactly-once delivery" unqualified; the lib has three at-least-once paths. slug:d7-hex-description | The Hex blurb qualifies exactly-once to the sink-owned/transactional path (Critical Rule 3) | — | Honesty on the most-visible surface |
| D8 | Foundational ADRs + published Critical Rules — docs/adr/ has only 0001/0002; the load-bearing posture decisions (value-free boundary, commit-LSN transaction watermark, spill-as-ephemeral-scratch, :one_for_all+:temporary supervision) are unrecoverable from code alone; the 5 Critical Rules live only in the un-tarballed AGENTS.md. slug:d8-foundation-adrs | Four ADRs land under docs/adr/; a tarball-shipped invariants doc (in mix.exs extras) carries the 5 Critical Rules for a Hex consumer | — | A 1.0 is the moment a bare-clone maintainer must recover the WHY |
| D9 | Three vendored public functions unspecced — Casting.Types.cast_record/2 (the central casting contract), Casting.ArrayParser.parse/1, Decoder.OidDatabase.name_for_type_id/1. slug:d9-vendored-specs | All three carry @spec; the frozen public surface is fully specced; dialyzer clean | — | Freeze the contract |
| D10 | Named-ctrl-conn cascade masked the integration suite — every integration module started a NAMED Postgrex pool in setup and never stopped it; ExUnit's async: false one-process model then made test 2+ fail with {:already_started, _}, silently masking ~half the suite (the full run was 66/31). Found while verifying D5. slug:d10-named-conn-cascade | PG16.named_conn/2 centralizes per-test isolation (start unlink + register on_exit stop); all 23 named-pool sites route through it; the full integration suite is 66/0 (was 66/31) | D5 | A masked integration suite is not 1.0-grade correctness evidence |
Status (2026-08-13 closeout)
All ten rows shipped in 1.0.0 (v1.0.0): D1 (install constraint), D4–D10 closed in the
hardening run, and the two 2026-08-13 major-contract rows — D2 (actual-session identity:
Replicant.SessionIdentity, handle_session_identity/2, the IDENTIFY_SYSTEM connect step,
ADR-0007) and D3 (release hygiene: .tool-versions pins Elixir 1.20.3-otp-29 / Erlang 29.0.3,
CI matches and uses immutable action/image revisions, mix audit is a gate, postgrex floors at
~> 0.22.4) — both verified against the fetched package. The only separately-authorized item is
the coordinated AshReplicant ~> 1.0 dependency publication. 1.1.0 (v1.1.0) is a
post-1.0 hardening patch: a float-array casting fix, a post-halt window-guard consistency fix,
and connection-opt merge hardening (see CHANGELOG [1.1.0]).
P2 backlog (post-1.0 hardening)
Not blocking the freeze; tracked here so nothing is lost. ✅ = shipped in the 1.0-hardening run.
- ✅ Reader "exactly-one" invariant — structural guard (was comment-defended):
Incremental.register_reader/1registers under{:incremental_reader, slot}inReplicant.Registry; a forgottenretire_reader/1halts fail-closed (:duplicate_reader) instead of double-delivering.37ab768. - ✅
assembler.exgod-module split DONE — extractedAssembler.Streaming(proto-v2 streaming + spill, 398 LOC) andAssembler.Batch(batched-checkpoint buffering + flush, 230 LOC); Core is now 1072 LOC (was 1633),%Assembler{}struct UNCHANGED. Not mechanical — the move carries a cross-cutting Rule-1 scrub cluster (every sink-call site keeps its value-freerescue/catch), a runtime-resolved Core↔Batch call cycle (Core routes to Batch; Batch calls Core'sreset/span_base/safe_shape/write_checkpoint), and 8 made-definternal seams (@doc false); it is the lowest-risk path to review-fidelity — pure functions on an unchanged struct, verified green at every commit by the 545-unit + 613-integration preservation net (no behavior change). Closeout HEADf99ce14(range481d54d..f99ce14, 7 commits): forge T2; fresh-context diff-review CLEAN; cross-vendor codex+claude CLEAN (one stale-comment fix landed); crash-injection marquees loss=0 / effect-dup=0 green against live PG16. - B8 checkpoint-read cache —
Assembler.skip?/2does a livesink.checkpoint()/0read per Commit (sink-owned clause inlib/replicant/assembler.ex); the throughput floor in sink-owned mode. Profiling-gated. - ✅ Duplicated batch flush-trigger
cond— extracted toAssembler.maybe_trip_batch/3(lib-batch and sink-owned-batch trip identically; no drift).55316e9. - ✅ Spill cleartext-at-rest threat model — documented in ADR-0005 (D8) + README "Operator guidance".
Replicant.Config.t()is a map, not a struct — ✅ frozen:@typedocstates it is a plain map by contract (built only byvalidate/1; adding keys allowed, renaming/re-typing is 2.0).d0ebed4.handle_batch/1arity — ✅ frozen:@docstates arity-1 is the 1.0 contract (no context arg; the batch high-water isList.last(transactions).commit_lsn; a context arg is a 2.0 callback change).d0ebed4.- ✅ Conformance tamper-evidence — machine-checked: a parametric byte-flip test (type byte + sampled payload) per message class proves each fixture goes red on mutation.
b04b4d7. - ✅ (corrected) Effect-once / loss=0 / §4 unit proof — the readiness test-audit over-claimed "zero unit-level proof"; probing the suite shows the invariants ARE unit-covered: watermark-skip (
assembler_test.exs:188-246), batch write-fault + watermark discipline (:791-800),reset_batch/reconnect-discard (assembler_server_test.exs:405-470), the §4:sink_too_slowhalt + first-frame no-false-halt (connection_test.exs:295-362), idle-ack ordering (:200). Crash-injection is integration-only by nature (the logic is unit-tested; the end-to-end marquees prove it under real reconnect). No gap to close. - ✅
Replicant.SnapshotProgressungrouped in HexDocs + three public fns had@specbut no@doc— SnapshotProgress grouped under "Data structures";@docadded toReplicant.Supervisor.start_link/1,Telemetry.span/3,Telemetry.event/3.d0ebed4. - ✅ Telemetry test-noise — silenced the
:telemetryattach-time:infoadvisory by setting the test Logger level to:warning(one line; replicant emits no Logger calls, so nothing useful is hidden).a6f39f5.