Changelog

All notable changes to reckon-db will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[5.9.0] - 2026-07-04

Added

• Liveness watchdog for a hung local ra server (Tier-2 resilience). A hung gen_statem never crashes, so OTP supervision never restarts it — a wedge could linger. The healer now tracks consecutive unresponsive audits and, when the local server is unresponsive with NO majority to rejoin from (a majority triggers the orphan force-delete+rejoin path instead), force-recycles it after ?UNRESPONSIVE_RECYCLE_STREAK audits with a preserve-data restart (ra:stop_server + ra:restart_server, timeout-guarded), which replays the durable log and un-wedges a hung-but-intact server. It never wipes here — with no majority there is nothing to re-replicate from — so a preserve-data restart is the correct, non-destructive remedy. New wedged_local verdict; telemetry via the existing heal events; last_action => recycled.

[5.8.3] - 2026-07-04

Fixed

• The coordinator can no longer wedge either. 5.8.2 made the healer wedge-proof; the coordinator's is_multi_member/1' still judged health viakhepricluster:members' + reckon_db_cluster:health_check' — both an unboundedstatem_call' into the local ra server. That is why the coordinator itself froze on the wedged store (and needed the healer, or a human, to recover). It now uses the new reckon_db_cluster:local_healthy/1', a wedge-proof predicate built on the lock-freera_leaderboard' ETS table plus a bounded ra:members/2' liveness probe. A wedged/isolated/split replica reads as unhealthy and keeps reconciling instead of blocking forever, so the whole "stuck orphan needs manual recovery" class is closed at the source. ### Added -reckon_db_cluster:local_healthy/1— wedge-proof local health predicate (leader present + locally clustered with it + local server responsive within a hard bound), for hot-loop health polling by the coordinator and healer. ## [5.8.2] - 2026-07-04 ### Fixed - **The healer no longer wedges on the very server it heals.** Live diagnosis of a stuck replica (run queue 1 — idle, NOT CPU-bound) showed both the coordinator and the healer frozen inkhepri_cluster:do_query_members -> ra_server_proc:statem_call: an unbounded synchronous call into the local ra server, which was itself wedged (member queries accepted, never answered). A monitor that calls the monitored server's synchronous API inherits its wedge, so the healer never completed a single audit (heal_countstuck at 0). - Fact-gathering is now entirely **lock-free / hard-bounded**: local + peer membership come from thera_leaderboardETS table; ara:members/2liveness probe (2s bound) tells us whether the local server is *responsive*. The audit never callskhepri_cluster:members/get_quorum_status. - **A wedged local server is now itself a healable fault**: newlocal_responsivesignal — if the local ra server can't answer within the bound while a majority exists, reset+rejoin (previously only a diverged member set triggered healing). - **Corrective action no longer needs the sick server's cooperation**: the gracefulkhepri_cluster:reset(a statem call that hangs on a wedged server) escalates tora:force_delete_server/2, which tears the server + data down unilaterally, then restarts fresh and rejoins. -reckon_db_store:ra_system_name/1exported for the force-delete. ### Added -ra_system_name/1toreckon_db_store's public API. ## [5.8.1] - 2026-07-04 ### Fixed - **Healer now correctly heals the diverged singleton** — the exact split it exists to fix. Two bugs in the 5.8.0 orphan detection: 1. It short-circuited onself_status = healthy, but a replica that split into its own 1-node cluster is LOCALLY healthy (quorum 1/1, leader = self), so it never acted. Orphan detection no longer keys on local self-health. 2. It then keyed on whether we appear in the MAJORITY's member list — but the majority still lists a diverged node as a configured-but-lagging member, so it counted the orphan as present and did nothing. Orphan detection now keys on our OWN LOCAL view: is the majority's elected leader absent from the cluster we are locally part of? A reset singleton has local members = [self] (leader absent -> orphan, healed); a node merely partitioned from a cluster it is still configured in keeps the full local set (leader present -> left to Ra). Regression testsclassify_healthy_singleton_is_orphaned+ghent_partition_is_not_reset. ## [5.8.0] - 2026-07-04 ### Added - **Continuous cluster self-healing.** A new per-storereckon_db_store_healercloses the detect→heal loop: it runs an always-armed periodic audit and, when this replica has drifted out of its store's quorum-holding cluster (the classic deploy join-race that left a replica an orphaned singleton), it automatically resets the local diverged Ra/Khepri state and rejoins the majority — no operatorwipe-and-rejoinscript needed. - **Data-safety gate** (safe_to_reset/1): a destructive reset is permitted ONLY for a replica the majority has never accepted — never the leader, never a member of the majority, and only when a real (leader-having) majority exists to rejoin. Transient partitions of a real member are left to Ra. - **self_heal => auto | alarm_only** store-configoptionsknob (defaultauto) for operators who prefer detect-and-alarm without destructive action. - **Observability:** new telemetry events[reckon_db, cluster, drift, detected],[reckon_db, cluster, heal, started|succeeded|failed|blocked],[reckon_db, cluster, reset, performed];reckon_db_cluster:health_check/1now returns aself_healingsection (mode,heal_count,last_heal_at,last_action,last_verdict) for admin dashboards. ### Fixed - **Coordinator no longer latches "joined" on a false-positive local view.**khepri_cluster:members/1returns the CONFIGURED member set (still 3 even when this node is isolated in a minority), so the oldlength(Members) > 1health test declared a partitioned replica healthy and stopped reconciling forever. Health is now judged authoritatively viareckon_db_cluster:health_check/1(reachability-based quorum + an elected leader), so an isolated/split replica keeps reconciling and reacts to peers reappearing. ## [5.7.0] - 2026-07-04 ### Added -reckon_db_integrity_key:status/1— the store's public integrity advertisement (#{enabled, algo, key_id}; never the key bytes). reckon-db now owns the algorithm id + key id (previously hardcoded in the gateway), so a remote gateway can dispatch for a store's integrity status via the{integrity_status, StoreId}gateway-worker call instead of guessing from its own local state. ### Changed - Requires **reckon-gater~> 3.10** (3.10.0routesintegrity_statusso the new worker call is reachable through the gateway on the same node). ## [5.6.1] - 2026-07-04 ### Fixed - Store-cluster join-reset race: a replica that elected to JOIN could get stuck forever on"Local Ra server ... is not registered", never converging (seen live as a permanent 2-of-3 store, surviving even a clean wipe + redeploy).khepri_cluster:joinresets the local store as part of joining; a join interrupted mid-reset (the coordinator's timeout guard kills it) left the local Ra server torn down, and the retry loop only ever re-tried the join (which needs a local server) without restarting the store. -reckon_db_store:ensure_khepri_started/1idempotently (re)starts the local Khepri/Ra server via the surviving store worker. -reckon_db_store_coordinatornow self-heals the local store before retrying a join, so a torn-down replica recovers instead of looping. - Regression suitereckon_db_store_heal_SUITE. ## [5.6.0] - 2026-07-04 ### Added -reckon_db_streams:global_event_count/1— an O(1) read of a store's total event count. A single monotonic counter node ([metadata, stats, global_event_count]) is incremented by every append batch inside the append's existing Khepri transaction (mirrors the DCB sequence counter), so the count is exact and adds no extra Ra round-trip. This replaces a full-store scan for ingest-rate/activity dashboards. - Exposed to gateways via the{global_event_count, StoreId}gateway-worker call. - Note: the counter starts at 0 on stores that predate it; it counts appends from that point on (a rate is correct immediately; a lifetime total is not backfilled). ### Changed - Requires **reckon-gater~> 3.9** (3.9.0routesglobal_event_countso the new counter is reachable through the gateway on the same node). ## [5.5.5] - 2026-07-03 ### Fixed - Store-cluster split-brain on simultaneous cold boot.reckon_db_store_coordinatorelected its coordinator over ALL connected nodes, so on a dist mesh that runs many single-store nodes (e.g. 12 nodes, one tenant store each) the globally-lowest node NAME — which may not run the store — was elected and every join for the store failed forever, leaving the replicas as separate 1-member clusters. The election now runs only over nodes that actually run the store (store_runner_nodes/2+ a pure, unit-testedelect_coordinator/2). In addition, a self-elected coordinator now keeps reconciling (in cluster mode) until it is genuinely multi-member instead of stopping the moment it returnscoordinator, so a peer that self-elected on a partial boot view, or a lower store-runner that connects late, still converges to one cluster. Restores the persistent-reconcile behaviour of the ex-esdb predecessor. Seeplans/PLAN_FIX_STORE_CLUSTER_SPLIT_BRAIN.md. ## [5.5.4] - 2026-07-02 ### Added — Telemetry guide - Newguides/telemetry.md: the full ReckonDB telemetry event catalogue (stream, subscription, snapshot, store, cluster, emitter, temporal, scavenge, causation, schema, memory-pressure, consistency, health, and link events) organised by category with each event's Measurements/Metadata contract, plus how to attach the built-in logger handler, custom handlers, and metrics exporters (Prometheus / OpenTelemetry). Documents the[reckon_db, cluster, leader, elected]event as the recommended follow-the-leader hook, and notes that the facade'sall_events/0covers the core subset (attach directly to the extended events by name). Wired into the hexdocs sidebar. Documentation only — no code change. ## [5.5.3] - 2026-07-01 ### Fixed — emitter teardown crash and event-type-summary worker crash -reckon_db_emitter_group:leave/3matched onlypg:leave/3 -> ok, butpg:leave/3returnsnot_joinedfor a pid that was never a member (an emitter terminating before it joined, or a double-leave on teardown). That benign no-op became a{badmatch, not_joined}in the emitter'sterminate/2and ashutdown_errorin the emitter-pool supervisor. Now tolerates both. -reckon_db_store_inspector:count_types_in_stream/3calledreckon_db_streams:read_all/4as(StoreId, StreamId, forward, 10000), but the arity is(StoreId, StreamId, BatchSize, Direction). The swap fedcalculate_versions/3Count = forward/Direction = 10000, afunction_clausethat crashed the gateway worker on everyGetEventTypeSummaryand — via the gateway retry loop — timed out the gRPC call. Arguments corrected to(10000, forward). Both found via the reckon-dotnet E2E suite. ## [5.5.2] - 2026-07-01 ### Fixed — snapshot record/read dropped metadata and did not round-trip datareckon_db_gateway_workerforwarded the gateway's#{data, metadata, version}envelope straight intoreckon_db_snapshots:save/4as theDataargument, so the whole envelope was persisted asdataand the caller'smetadatawas dropped (save/4defaults metadata to#{}). A snapshot recorded with data{"balance":100}read back as{"data":...,"metadata":...,"version":3}. The worker now unwraps the envelope and usessave/5, so data and metadata land in their own snapshot fields; a bare term still falls through asdatafor older callers. Found via the reckon-dotnet snapshot E2E round-trip (reckon-dbissue #2). ## [5.5.1] - 2026-06-24 ### Fixed — CCC payload indexes silently empty for atom-keyed event dataextract_payload_entries/2looked up declared payload keys (binary, e.g.<<"lot_id">>) against the event'sdatamap withmaps:get/3, but event-sourced producers (evoq) build domain events whosedatais a map with **atom** keys and binary values (#{lot_id => <<"L1">>}). The binary key never matched the atom key, soby_payload/by_payload_hashindex entries were never written —tagsandevent_typeindexed fine (those come from the binary#eventfields), so the gap was invisible until a CCC payload/hash query returned nothing despite matching events existing.decode_json_map/1now normalises atom keys to binary, so the declared binary key matches regardless of whether the producer keyeddatawith atoms, binaries, or a JSON binary. The producer owns event content; the store indexes it as given. Covered byreckon_db_dcb_payload_extract_tests. ## [5.5.0] - 2026-06-24 ### Added — gateway worker payload index introspectionreckon_db_gateway_workernow handles{get_payload_indexes, StoreId}and{get_payload_hash_indexes, StoreId}requests from reckon-gater 3.7.0+. The worker reads directly from the#store_config{}it holds in its gen_server state (captured atstart_link/1time) — no Khepri I/O. The filters extract{payload, Key}and{payload_hash, Keys}entries from theindexesfield. Required byreckon_gater_api:get_payload_indexes/1andreckon_gater_api:get_payload_hash_indexes/1. ## [5.4.0] - 2026-06-23 ### Changed — CCC module rename Renames two modules to reflect that their scope is CCC (Command Context Consistency), not just the DCB subset. Storage-layer modules (reckon_db_dcb,reckon_db_dcb_paths) retain theirdcbprefix because the on-disk Khepri paths ([by_tag, ...],[by_payload, ...]) are unchanged — renaming them would break existing stores. -reckon_db_dcb_filter→ **reckon_db_ccc_filter** (deleted; module evaluates the full CCC tag_filter algebra including payload predicates). - Payload path builders extracted fromreckon_db_dcb_pathsinto new **reckon_db_ccc_paths** (by_payload_path/3,by_payload_pattern/2,by_payload_hash_path/2,by_payload_hash_pattern/1,payload_combo_hash/2).reckon_db_dcb_pathsretains only DCB-scoped functions (event path, tag paths, event-type paths, seq_key encoding). ### Added -reckon_db_ccc_filter` — CCC filter evaluation (matches all filter shapes: tags, event_type, payload_match, payload_hash_match_pre, and, or). - reckon_db_ccc_paths — Khepri path builders for payload indexes. - guides/ccc.md — reckon-db CCC guide covering store index configuration, filter shapes, the Horus constraint, and implementation module references. ### Docs - guides/dcb.md — TagFilter table extended from 5 to 7 shapes (added payload_match and payload_hash_match rows with index column); cross- reference to ccc.md added. - guides/dcb_raft_design.md — code example updated to reckon_db_ccc_filter, try_append/7 arity corrected, payload index paths added to the index table, CCC guide cross-reference added. ### Upgrading from 5.3.0 Replace all references to reckon_db_dcb_filter: with reckon_db_ccc_filter: and all references to `reckon_db_dcb_paths:by_payload/reckondb_dcb_paths:payload_combo_hashwith the equivalent functions inreckon_db_ccc_paths. On-disk Khepri state is unchanged; no store migration required. --- ## [5.3.0] - 2026-06-23 ### Added — CCC payload indexes (Command Context Consistency) Extends the DCB conditional-append primitive with two payload-indexed filter variants from reckon-gater 3.5.0, completing Command Context Consistency (CCC) support. Events are now indexed by declared JSON payload fields as well as tags and event types, enabling consistency boundaries on payload data without requiring writers to anticipate future queries as tags. **New index declarations** (store_config.indexes): -{payload, Key :: binary()}— indexes the top-level JSON fieldKeyfromevent.data. Values must be binary (string). Absent or non-binary values produce no index entry. -{payload_hash, Keys :: [binary()]}— indexes a combination of fields as a SHA-256 hash. Enables{payload_hash_match, Keys, Values}queries with O(1) Khepri path length regardless of field count. **New filter variants** (handled inappend_if_no_tag_matchesand reads): -{payload_match, Key, Value}— matches events wheredata[Key] = Value. -{payload_hash_match, Keys, Values}— matches events where the composite of all listed field values matches (field order is ignored; hash is pre-computed outside the transaction to respect the Horus extraction constraint oncrypto:hash/2). **New read functions** inreckon_db_dcb: -read_by_payload/4— reads events by single-field payload value. -read_by_payload_hash/4— reads events by composite payload field combination. **New helper** (inreckon_db_ccc_pathsas of 5.4.0): -reckon_db_ccc_paths:payload_combo_hash/2— SHA-256 of a sorted[{Key,Value}]list. Order-independent. Must NOT be called from inside a Khepri transaction (NIF constraint). **Implementation notes**: - All payload writes happen inside the existing conditional-append Khepri transaction, atomically with the event record and tag/event-type index entries. - Hash pre-computation usespreprocess_filter/1inreckon_db_ccc_filter(wasreckon_db_dcb_filterin 5.3.0; renamed in 5.4.0), invoked outside the transaction byappend_if_no_tag_matches. - Thereckon_db_indexsecondary index is unaffected:{payload, }and{payloadhash, }declarations return[]fromentriesfor/4. - Stores without any{payload, }or{payloadhash, }declarations incur zero overhead —declared_dcb_payload/1returns[]on every append and the payload extraction path is skipped. - Requires reckon-gater ~> 3.5 (for the newtag_filter()variants). **Tests**: 69 new unit tests acrossreckon_db_dcb_paths_tests,reckon_db_dcb_filter_tests, andreckon_db_index_config_tests(modules renamed tocccin 5.4.0). ## [5.2.2] - 2026-06-23 ### Added — DCB read APIs: log, tags, event-types Three new functions inreckondb_dcb: -read_log/3— paginated sequential read of DCB events (FromSeq,Limit). Returns{ok, Events, TotalCount}whereTotalCountis the full log size. -all_tags/1— enumerate all tags in the by_tag index with event counts, sorted descending. Returns{ok, [{Tag, Count}]}. -all_event_types/1— enumerate all event types in the by_event_type index with event counts, sorted descending. Returns{ok, [{EventType, Count}]}. Gateway worker gains correspondingdcb_read_log,dcb_all_tags,dcb_all_event_typeshandle_call clauses. These back the new admin UI DCB views (Log, Tags, Event Types) added in reckon-gateway 0.12.0. ## [5.2.1] - 2026-06-23 ### Fixed — Khepri startup race:{ok, {error, noproc}}crash In Khepri 0.17.x,readwrite_transaction1has a catch-all clauseRet -> {ok, Ret}that wrapsprocess_commandfailures (e.g.{error, noproc}when Ra is not yet ready during store startup) as{ok, {error, noproc}}. Neitherreckon_db_streams:write_batch/3norreckon_db_dcb:try_append/6handled this shape, causing gateway workers to crash and retry-loop on first writes after a node restart. Added{ok, {error, E}} -> {error, E}clause to both paths. The DCB path also now guards{ok, LastSeq}withis_integer(LastSeq)to prevent a silent wrong-type success if this race recurs. ## [5.2.0] - 2026-06-22 ### Added — DCB event-type filter and[by_event_type]index Closes the spec-conformance gap with the canonical DCB specification (dcb.events):tag_filter()now supports an{event_type, binary()}leaf that matches events by theirevent_type` field, composable with the full boolean algebra. New filter shape (requires reckon-gater 3.4.0+): ```erlang {event_type, <<"user_registered_v1">>} %% Compose freely with tags — this is the canonical DCB "query item": {and, [ {eventtype, <<"user_registered_v1">>}, {any_of, [<"email:alice@example.com">]} ]} `` **Storage:** Each DCB event is now indexed at[by_event_type, EventType, SeqKey]alongside the existing[by_tag, Tag, SeqKey]entries. The index is written atomically inside the same Khepri transaction as the event itself. **Evaluation:**reckon_db_dcb_filter:match_seqs/2handles the new{event_type, T}clause;match_any_above_cutoff/2uses a newdefault_provider/1that dispatches toseqs_for_tag/1for binary keys andseqs_for_event_type/1for{event_type, T}keys. Both follow the same O(bounded-subtree) pattern as the tag lookup. **Migration:** Existing DCB events written before 5.2.0 have no[by_event_type]index entries.{event_type, T}filters will not match pre-existing events. If you need full coverage, re-write those events throughappend_if_no_tag_matches/4or accept that the cutoff for pre-5.2.0 events is effectively -1 for type-based queries. **Backward compatible:** All existing{any_of, ...},{all_of, ...},{and, ...},{or_, ...}filters are unchanged. ## [5.1.0] - 2026-06-10 ### Security — multicast discovery hardened (gossip protocol v2) Fixes the 2026-06-10 audit finding: the discovery listener fed attacker-controllable UDP multicast datagrams straight into an unsafebinary_to_term/1` before any authentication (remote atom-table exhaustion + unbounded allocation from any LAN host), the cluster secret rode the wire in cleartext, and an unconfigured deployment silently fell back to the hardcoded reckon_db_default_secret. - Safe decode first. Inbound datagrams are decoded with binary_to_term(_, [safe]) and pattern-matched against the exact v2 shape; anything else is dropped. - HMAC instead of cleartext secret. v2 gossip is {gossip_v2, NodeBin, Timestamp, Hmac} with HMAC-SHA256 keyed by the cluster secret (constant-time compare, ±60s freshness window). The secret never goes on the wire. Node names travel as binaries and are only atomized after authentication. - No default secret. Cluster-mode discovery now requires RECKON_DB_CLUSTER_SECRET (or the new {reckon_db, cluster_secret} app env) and stays passive with a loud error log when unset. Manual/static cluster joins are unaffected. Rolling-upgrade note: v1 and v2 gossip are mutually unintelligible. During a mixed-version window, old and new nodes will not discover each other via multicast; upgrade all cluster nodes, or join explicitly. Deployments that relied on the implicit default secret must now set one. New test module: reckon_db_discovery_gossip_tests (roundtrip, secret-not-on-wire, wrong secret, tampered name, stale timestamp, legacy v1, garbage, unknown-atom ETF payload, oversized node name). ## [5.0.0] - 2026-06-08 ### Changed — structural aggregate-type stream namespace (Model C) — BREAKING (on-disk layout) Events are now stored under a structural aggregate-type subtree, [streams, Type, Id, Version], instead of the flat [streams, StreamId, Version] layout. The aggregate type (the user-id prefix, or the $ namespace for system streams) is a real Khepri path level, so type-scoped operations (list_streams, replay, links) navigate a subtree instead of scanning the whole store. The Erlang API is unchanged (reckon_db_streams function signatures are identical; stream ids are still opaque <prefix>-<hex> / $ns:name). The break is the on-disk Khepri layout: there is no migration — recreate stores fresh under the new layout (reckon-db carries no production data; this is the project's deliberate clean-break policy). reckon_db_stream_path is the single owner of the layout. Requires reckon_gater ~> 3.1 (uses reckon_gater_stream_id:parts/1). ### Added — generic write-maintained secondary index (opt-in per store) Cross-cutting lookups (by tag, by event type, by metadata key=value) become O(matches) subtree reads instead of O(total events) scans, for stores that declare the index. Reference entries are maintained transactionally with each append (atomic — the index is never partially populated). - #store_config.indexes :: [index_decl()] (default []) — declare tags, event_type, and/or {meta, Key}. A store pays nothing unless it declares an index. - New reckon_db_streams:read_by_metadata/3 (+ reckon_gater_api:read_by_metadata/3 in reckon_gater 3.2.0) — the sanctioned primitive applications build causation/correlation/saga read models on. The store does not interpret the key; lineage traversal is the application's job. - read_by_tags/4 and read_by_event_types/3 now use the index when declared and fall back to a (warned-once) whole-store scan otherwise. - Index layout lives under a dedicated [idx] root, separate from DCB's seq-keyed [by_tag] (DCB is untouched). - No backfill / building-ready marker in this release — declare indexes at store creation; recreate the store to add one (not in production). ### Changed — append is now transactional The append write path writes a batch's event records (and any declared index entries) in a single khepri:transaction, so a multi-event batch and its index entries land atomically. Integrity (HMAC + chain) stamping continues to happen outside the transaction (it needs the key from persistent_term and crypto, neither available inside a Ra transaction) — the records are stamped first, then written, mirroring the DCB append. ### Fixed - by_stream subscription filter now matches exactly the subscribed stream's `[streams, Type, Id, ]subtree (Model C made the type structural, resolving the prior over-matching "by_stream path-mismatch" behaviour). -subscribe/5rejects a duplicate subscription whose subscriber is still alive with{error, {alreadyexists, Name}}; a dead/undefined subscriber pid still reclaims via the reconnect path. ## [4.0.0] - 2026-06-07 ### Removed — causation/correlation service + graph NIF (BREAKING) Causation/correlation lineage traversal is not an event-store responsibility — it's a read-model / projection concern (the way EventStoreDB serves it via system projections, not log scans). Removed: -reckon_db_causation(theget_effects/get_cause/get_chain/get_correlated/build_graphfull-scan queries) -reckon_db_graph_nifand its entire Rust crate (native/reckon_db_graph_nif/, petgraph-based graph builder) - the gateway-workerhandle_callrouting for those operations -guides/causation.md,assets/causation_graph.svg, and the causation/graph_nif test modulescausation_idandcorrelation_idare unchanged — they remain ordinary keys inside an event'smetadatamap (see the metadata examples inguides/event_sourcing.md). The store stores and returns metadata verbatim; it does not interpret it. Consumers that need to traverse lineage build a read model/projection. Requiresreckon_gater ~> 3.0(which drops the matching API surface). Also reverts the 3.1.x causation-scan perf workaround (all_events/1), moot now the module is gone. ## [3.1.2] - 2026-05-30 ### Fixed — Transient store-not-ready crashed the gateway worker into a permanent outagereckon_db_streams:append_events_to_stream/4matchedok = khepri:put(...)as a hard assertion. Whenkhepri:putreturned{error, noproc}— most commonly while the store's Ra server was mid-(re)start — the badmatch escaped throughreckon_db_gateway_worker:handle_call/3and **crashed the gateway worker**. Under a boot-time append (a producer firing before Ra is write-ready) the worker crash-looped past its supervisor's restart intensity in under a second; the cascade reachedreached_max_restart_intensityand tore down the **entire store subtree** ("Khepri store stopped"). A sub-second readiness gap thus became a **permanent, restart-looping outage** — every subsequent append failednoproc, and the producer's retry storm formatted giant error reports that pegged the CPU.append_events_to_stream/4now returns{error, Reason}on a failedkhepri:putinstead of badmatching (spec widened to{ok, non_neg_integer()} | {error, term()}). The gateway worker already replies{error, }cleanly, andreckongatertreats it as retriable — so an early append now retries and succeeds once Ra is ready, instead of killing the store. Diagnosed on the parksim fleet: the leuven store had been restart-looping, persisting no new events while the SQLite read model sat frozen. ## [3.1.1] - 2026-05-27 ### Notes — Why 3.1.1 and not 3.1.0 3.1.0 was tagged locally but never published to hex: a parallel docs fix shipped after the tag was cut, and thereckon_gaterconstraint was tightened to~> 2.3.0(only 2.3.x). Both issues are folded into 3.1.1, paired withreckon_gater 2.3.1. ### Changed —reckon_gaterconstraint widenedrebar.config:{reckon_gater, "~> 2.3.0"}→{reckon_gater, "~> 2.3"}. Allows coordinated minor-version updates across the stack without re-tagging consumers. Per workspace convention (MEMORY.md): use loose~> X.Yconstraints; exact-patch pins block coordinated library updates. ### Fixed — EDoc-incompatible backticks across DCB modules DCB module + function@docblocks used markdown-style backticks for inline code references, which EDoc's parser rejected with ``-quote ended unexpectedly `. Replaced with plain text;rebar3 ex_docnow builds clean. Affectsreckon_db_dcb,reckon_db_dcb_paths,reckon_db_dcb_filter,reckon_db_streams. ### Added — DCB (Dynamic Consistency Boundary), Phase 3 first slice Query-based conditional-append primitive that complements the existing stream-version optimistic concurrency model. Where a Dossier aggregate locks on its own stream's version, a DCB Decision locks on the absence of new events matching a tag-filter context query. Seeplans/PLAN_DCB_IMPLEMENTATION.mdfor the full design rationale. **New public API** (reckon_db_streams): ```erlang %% Conditionally append events under the DCB pseudo-stream. %% Returns {error, {context_changed, MaxSeq}} when any event matching %% TagFilter has seq > SeqCutoff; nothing is written in that case. reckon_db_streams:append_if_no_tag_matches( StoreId, TagFilter, SeqCutoff, Events). ```TagFiltersupports{any_of, [Tag]},{all_of, [Tag]},{and, [Filter]},{or, [Filter]}with per-event semantics. **New modules**: -reckon_db_dcb— thekhepri:transaction/2-based primitive -reckon_db_dcb_paths— Khepri path helpers + zero-padded decimal seq keys -reckon_db_dcb_filter— pure tag-filter algebra over a mockable seqs provider **New behaviour callback** onreckon_db_log_backend: -append_if_no_tag_matches/5— optional. Backends that don't implement it return{error, not_supported}at the gateway layer. **New macros** ininclude/reckon_db.hrl: -?DCB_STREAM=<<"_dcb">>— pseudo-stream id for DCB events -?DCB_STREAM_PATH=?STREAMS_PATH ++ [?DCB_STREAM]— DCB events live under[streams, "_dcb", SeqKey]so all existing read paths (read_all_global,read_by_event_types,read_by_tags, subscriptions) see them automatically -?BY_TAG_PATH=[by_tag]— root of the tag index -?DCB_SEQ_COUNTER_PATH=[metadata, dcb, last_seq]— global counter -?DCB_SEQ_KEY_WIDTH= 20 — decimal padding for seq keys ### Added — DCB gateway dispatch (P3.4)reckon_db_gateway_worker:handle_call/3now handles{append_if_no_tag_matches, StoreId, TagFilter, SeqCutoff, Events}requests routed fromreckon_gater_api:append_if_no_tag_matches/4(reckon-gater 2.3.0+). End-to-end transport path: ``` caller -> reckon_gater_api:append_if_no_tag_matches/4 -> reckon_gater_api:route_call/2 -> gen_server:call(worker_pid, {append_if_no_tag_matches, ...}) -> reckon_db_gateway_worker:handle_call/3 -> reckon_db_streams:append_if_no_tag_matches/4 -> reckon_db_dcb:append_if_no_tag_matches/4 -> khepri:transaction/2 body ``` The direct in-process path (reckon_db_streams:append_if_no_tag_matches/4→reckon_db_dcb) continues to work unchanged for callers inside the BEAM node. ### Changed —tag_filter()andseq_cutoff()types consolidated The canonical home fortag_filter()andseq_cutoff()is nowreckon_gater_types.hrl(reckon-gater 2.3.0+). The local definitions inreckon_db_dcb_filterare removed. Specs inreckon_db_dcb,reckon_db_streams, andreckon_db_dcb_filterreferencereckon_gater_types:tag_filter()/reckon_gater_types:seq_cutoff()directly. No semantic change; type re-homed for cross-repo sharing. ### Changed —reckon_gaterdep constraintrebar.configbumped from{reckon_gater, "~> 2.2.0"}to{reckon_gater, "~> 2.3.0"}. Required for the DCB types and theappend_if_no_tag_matches/4wire API. **Publish ordering**: reckon-gater 2.3.0 must be on hex before reckon-db 3.1.0 can be cleanly hex-published. For local development, add_checkouts/reckon_gatersymlinked to the local reckon-gater checkout (gitignored, so this is per-developer setup). ### Added — DCB integrity (HMAC chain for DCB events) DCB events on integrity-enabled stores now carryprev_event_hash+macand link into a per-store DCB chain rooted at the genesis hash. The fail-closed safety check ({error, integrity_not_supported_in_dcb_v1}) is replaced with real integrity support. Tampering with stored DCB events is detectable via the existingreckon_gater_integrity:verify_event/3read-path. **Implementation note:** Khepri's Horus extractor rejects code that containscrypto:*calls (build_stacktrace instructions in surrounding error handling). MAC chains are therefore pre-computed OUTSIDE thekhepri:transaction/2body, with the transaction verifying the chain-tip + counter are still what the caller observed. Two retry vectors: -{context_changed, }— tag-filter conflict (existing) -{dcbstate_changed, }— counter or chain-tip moved between snapshot and tx; retry handled internally with budget?INTEGRITYRETRY_BUDGET(5). Exceeded →{error, dcb_concurrent_writer_exhausted}. **New macro**?DCB_CHAIN_TIP_PATH = [metadata, dcb, chain_tip]holds the chain-hash of the last DCB event written under integrity. Absent path = no integrity-bearing DCB events yet; next write usesreckon_gater_integrity:genesis_prev_hash(). ### Notes — DCB v1 limitations remaining **Tag index is forward-only**: events written before this release do NOT have/by_tag/{Tag}/{SeqKey}mirror entries. They remain queryable via the existingread_by_tags/4(full-scan + filter) but are invisible to the DCB consistency check. This is correct for DCB consistency (only Decisions need consistency with other Decisions); existing aggregate-stream reads are unaffected. **Snapshots**: no snapshot support for the DCB pseudo-stream. Aggregate snapshotting doesn't apply (no per-aggregate consistency boundary). **Wire / Evoq integration**: theappend_if_no_tag_matchesfacade inreckon_db_streamscurrently callsreckon_db_dcbdirectly. The gateway worker dispatch (P3.4) and theevoq_decisionbehaviour in evoq (P3.6) ship in subsequent PRs per the plan. ## [3.0.0] - 2026-05-26 ### Removed (breaking) —reckon_db_stream_idmodule The stream-id format validator moves toreckon_gater_stream_id(in reckon-gater 2.2.0). Stream-id format is a protocol contract; it belongs in the gateway layer where both reckon-db (write validation) and reckon-evoq (adapter generation) can reach it without dragging reckon-db's khepri/Ra payload into pure-routing consumers. Callers using the old module must migrate: ```diff - reckon_db_stream_id:validate(StreamId) + reckon_gater_stream_id:validate(StreamId) ``` The samevalidate/1,is_valid/1, andis_system/1functions are exported fromreckon_gater_stream_idwith identical semantics, plus a newnew/1generator andprefix_of/1/suffix_of/1parsers. ### Changed (breaking) — user-stream regex tightened Inherited from reckon-gater 2.2.0. User-stream regex tightens from^[A-Za-z]+-[A-Fa-f0-9]+$` to `^[a-z]{1,32}-[a-f0-9]{32}$: - Prefix is now lowercase only, capped at 32 chars. - Suffix is now exactly 32 lowercase hex chars (128 bits). Existing data with non-conforming ids remains readable.append/4rejects new events on non-conforming streams with{error, {invalid_stream_id, malformed_user_id, StreamId}}. No production deployments to migrate. ### Changed — deps bumped -reckon_gater~> 2.2.0— picks up the relocated stream-id module. ## [2.3.7] - 2026-05-18 ### Renamed —subscribe_duplicate_fails→subscribe_duplicate_is_idempotentThe test name dated from whensubscribe/5rejected duplicates. The implementation has been idempotent since the reconnect-path work (returns{ok, Key}on a second subscribe with the same name). 2.3.6 already updated the test body to assert idempotency; this release brings the name in line. ### Changed — deps bumped -reckon_gater~> 2.1.4— picks up theno_snapshotretry whitelist soScavengeDryRunon a stream without a snapshot surfacesInvalidArgumentinstead of timing out the gRPC deadline. ## [2.3.6] - 2026-05-18 ### Changed — gateway worker handlesremove_subscription+ack_eventas calls Mirror of the 2.3.5save_subscriptionconversion. Both used to be fire-and-forget casts that returned no signal to the gateway; nowhandle_callreturns the underlying store result. -remove_subscriptionreturnsokfor both genuine removal and the idempotent "not_found" case (removal is the desired terminal state). -ack_eventreturns the underlyingreckon_db_subscriptions:ack/4result;{error, {subscription_not_found, }}surfaces when acking against a removed subscription. Pairs with reckon-gater 2.1.3, where both gater APIs are nowroutecalland the new error tag is whitelisted as non-retriable. ### Fixed —subscribe_duplicate_failstest matches actual contract The test asserted{error, {already_exists, }}from a duplicatesubscribe/5, but the implementation has been idempotent (returns{ok, Key}viareregistersubscriber/4) since the reconnect-path work. Test updated to assert idempotency — same key on both calls. ### Fixed —reckon_db_integrity_key_testscreate their tmp dir Tests relied on/tmp/reckon_db_integrity_key_tests/existing.make_sealed_file/1now callsfilelib:ensure_dir/1before writing. ### Changed —reckon_gaterdep bumped to~> 2.1.3Picks up the newremove_subscription/ack_eventsync contract +{subscription_not_found, }retry whitelist. ## [2.3.5] - 2026-05-18 ### Changed — gateway worker handlessavesubscriptionas a callreckon_db_gateway_workerused to receivesave_subscriptionas a fire-and-forgethandle_cast. If the underlyingreckon_db_subscriptions:subscribe/5returned{error, }(most notably{invalidfilter, }from a malformed selector), the worker logged a warning and the gRPC client never knew — Subscribe "succeeded" while no events ever flowed. Nowhandlecall, returning the real result. The matching call in reckon-gater 2.1.2 propagates the{ok, Key} | {error, }to the gateway, which translates the error to gRPCInvalidArgument(reckon-gateway 0.4.10).{alreadyexists, Key}from the store layer is mapped to{ok, Key}— re-registering with the same name is idempotent (reregister_subscriber/4re-binds the pid and re-arms the trigger), so consumers don't see a misleading error for that expected reconnect path. Pin tightened:reckon_gater ~> 2.1.2(was~> 2.1.1). 618/618 eunit pass. ## [2.3.4] - 2026-05-18 ### Fixed — Validator errors no longer time out gRPC clients Pinsreckon_gaterto~> 2.1.1(was~> 2.0). 2.1.1 adds{invalid_stream_id, , }to the non-retriable allowlist inreckon_gater_retry:is_retriable_error/1. Without this fix the validator introduced in 2.3.3 worked correctly at the storage layer — but its error tuple was treated as transient by the retry layer, which burned through 10× exponential backoff (~30 seconds) before giving up. gRPC clients sawDeadlineExceededinstead of the realInvalidArgumentcause they were supposed to get. With 2.3.4, malformed appends fail fast (single call, no retry) and the gateway surfacesInvalidArgumentto the caller as designed. Verified live against the 4-node beam cluster. ## [2.3.3] - 2026-05-18 ### Added — Stream-id format validator (guards against malformed writes) Stores no longer accept malformed stream ids at append time. The validator is enforced at the head ofreckon_db_streams:append/4, so every write path (gateway, links, direct API) goes through the same gate. **Accepted formats** (seeguides/system_streams.mdfor the full rationale): - **User stream:**<prefix>-<hex>where prefix is[A-Za-z]+and hex is[A-Fa-f0-9]+. Example:account-018f6a7b8c9d4abc8901234567890abc. - **System stream:**$<namespace>:<name>where namespace is[a-z][a-z0-9-]*and name is[A-Za-z0-9][A-Za-z0-9.-]. Example:$link:high-value-orders`. **Rejected with `{error, {invalid_stream_id, Reason, StreamId}}`:** empty ids, non-binary inputs, mid-string `$(e.g.partition$XYZ`, `test$basic-stream), bare ids without a hex tail, and$`-prefixed ids that don't match the system format. The gateway maps these to gRPC `InvalidArgument`. The new module `reckon_db_stream_id` is the single source of truth for the rules; 38 unit tests cover the grammar. ### Fixed — Test fixtures (58 stream-id literals) Test suites that produced malformed ids have been cleaned up so they pass the validator: - `reckon_db_test_helpers:generate_stream_id/0` — now emits `test-<lowercase-hex-32>`, was `test$<uuid-with-dashes>. - Integration suites swept (49 literals):reckondb_snapshots,reckon_db_subscriptions,reckon_db_subscription_delivery,reckon_db_emitter_autostart,reckon_db_integrity_subscriptions,reckon_db_pg_scope,reckon_db_streams. Pattern<<"test$X-Y">>→<<"testXY-001">>(alpha-only prefix + hex tail). - The companionreckon-e2etorture suites (integrity_torture,multi_node,adapterswap_torture) use the same convention now —<<"partition$">>` → `<<"partition-">>` before concatenating the random hex nonce. ### Compatibility This is the **first release where appending a malformed stream id fails**. Existing stores with old malformed paths (`leader-kill$XYZ, etc) continue to read fine — the validator only gates new writes. Wipe + redeploy if you want them gone. If a downstream test suite generates non-compliant ids that weren't covered above, the fix is to mirror the helper change:<<prefix-lowercase>-<lowercase-hex>>>. Seereckon_db_stream_id:validate/1for the precise grammar. ## [2.3.2] - 2026-05-17 ### Fixed — Gateway-facing subscription, lag, and snapshot bugs Surfaced by the new reckon-go SDK exercising paths no Erlang consumer had previously hit. - **reckon_db_filters:by_stream/1** no longer requires a$separator in the stream id. The check rejected plain ids with{error, invalid_stream}; the gateway worker logged a warning and silently dropped the subscription, leaving the client waiting forever for deliveries. The path component is used verbatim; there was never a semantic reason for the restriction. - **reckon_db_store_inspector:subscription_lag/2** matched{ok, Sub}againstfind_by_name/2, butfind_by_namereturns{ok, Key, Sub}per its spec. Every successful lookup crashed the gateway worker withcase_clause, surfacing as gRPCInternal. Now matches the documented 3-tuple. Companion test mockreckon_db_store_inspector_tests:lag_calculation/0updated — it was returning the same wrong shape and was hiding the production bug. - **reckon_db_gateway_worker.read_snapshotwithVersion = 0** now falls back toreckon_db_snapshots:load/2("latest"). The gRPCSnapshotService.ReadSnapshotproto has no read-latest RPC; this lets clients ask for the most recent snapshot in a single round-trip instead ofListSnapshots+ReadSnapshot. ## [2.3.1] - 2026-05-17 ### Fixed — Embedded NIFs actually ship in the hex tarball reckon-db 2.3.0 published with afileslist living in the wrong block ({pkg, [...]}inrebar.configinstead of{files, [...]}insrc/reckon_db.app.src). rebar3_hex silently fell back to its default file glob, which excludednative/entirely. So 2.3.0 on hex contained zero Rust crate sources — consumers gotpriv/build-nifs.shwith nothing to build. 2.3.1 puts thefileslist in.app.srcwhere rebar3_hex reads it, matching macula's pattern. **rebar3 hex publish --dry-runnow reports:** ``` Included files: native/reckon_db_crypto_nif/{Cargo.toml,Cargo.lock,src/lib.rs} native/reckon_db_archive_nif/{Cargo.toml,Cargo.lock,src/lib.rs} native/reckon_db_hash_nif/{Cargo.toml,Cargo.lock,src/lib.rs} native/reckon_db_aggregate_nif/{Cargo.toml,Cargo.lock,src/lib.rs} native/reckon_db_filter_nif/{Cargo.toml,Cargo.lock,src/lib.rs} native/reckon_db_graph_nif/{Cargo.toml,Cargo.lock,src/lib.rs} priv/build-nifs.sh docs/{dialyzer-backlog.md, dialyzer-warnings-2.2.2.raw, genai-policy.md} CONTRIBUTING.md CODE_OF_CONDUCT.md ... ``` #### Consumer expectations clarified rebar3_hex automatically strips compiled.so/.dll/.dylibbinaries from the published tarball (a security + reproducibility measure that applies to every hex package). v2.3.0's CHANGELOG implied otherwise — that the prebuilt.sofiles would ship "so consumers without cargo still get acceleration". They don't. macula has always worked the same way: the published tarball contains only Rust source + thebuild-nifs.shscript. So: - **Consumers withcargoinstalled** get full NIF acceleration —rebar.config'spre_hooksinvokepriv/build-nifs.shwhich runscargo build --releasefor each crate duringrebar3 compile. - **Consumers withoutcargo** get a warning in their build output and silently fall back to the pure-Erlang implementations baked into eachreckon_dbnifwrapper module. Everything still works; acceleration is just absent. For reckon-gateway specifically: the Docker base image (erlang:27-slim) does NOT include Rust. Operators wanting NIF acceleration on the cluster will need to add a Rust install step to the gateway's Dockerfile builder stage (a 2.3.x follow-up release of reckon-gateway is the natural place for that change). ### Documentation -src/reckon_db.app.src: longerdescriptionfield that mentions the NIF acceleration and pure-Erlang fallback;linksextended withDocumentation(hexdocs) andChangelogentries. ## [2.3.0] - 2026-05-17 — UNUSABLE, superseded by 2.3.1 > ⚠️ **This release published without any Rust crate sources due to > a misplacedfileslist. Treat 2.3.0 as functionally equivalent > to 2.2.2 — bump straight to 2.3.1.** ### Added — Embedded Rust NIF acceleration reckon-db now ships its own NIF acceleration in-tree, modelled on macula's pattern. The six previously-separate Rust crates from the [reckon-nifs](https://codeberg.org/reckon-db-org/reckon-nifs) sidecar are absorbed intonative/andpriv/of this package: | Crate | Speedup | |-------|---------| |reckon_db_crypto_nif| Ed25519 verify, SHA256 — 3-5× | |reckon_db_archive_nif| LZ4 compression — 5-8× | |reckon_db_hash_nif| xxHash, FNV-1a — 10-15× | |reckon_db_aggregate_nif| Vectorised aggregation — 5-10× | |reckon_db_filter_nif| Regex/pattern matching — 3-5× | |reckon_db_graph_nif| Graph algorithms — 5-10× | #### How it works -rebar.configpre_hook invokespriv/build-nifs.shbefore Erlang compilation. The script runscargo build --releasefor each crate and copies the resulting.sointopriv/. - Build script is **idempotent** (skips.sofiles already present) and **tolerant** (logs a warning and continues if the Rust toolchain isn't installed — wrapper modules then use the Erlang fallbacks). - Prebuilt.sofiles are shipped in the hex package, so consumers withoutcargostill get acceleration. - Each wrapper module's-on_load(init/0)looks incode:priv_dir(reckon_db)for the.so, with a fallback tocode:priv_dir(reckon_nifs)so users still pinned to the legacy sidecar package keep working. #### Why this consolidation The previous reckon-nifs sidecar had three layered problems: 1. **Name drift.** Crates were renamedesdb → reckondbin v2.0.0 but therustler::init!macros inside each crate kept declaring the OLD module name. Soerlang:loadnif/2from the reckon-db wrappers refused to load with{bad_lib, "Library module name 'esdb_hash_nif' does not match calling module 'reckon_db_hash_nif'"}. 2. **Dead loader.** A centralreckon_nifs_loader:load_all/0setesdbloadedpersistent_term keys that nothing read — the actual loading happens in each wrapper's own-on_load, not from a central place. The loader'serlang:load_nif/2calls couldn't have worked anyway because NIFs can only be loaded into the module that owns the stub declarations. 3. **Cross-application priv lookup.** The fallback tocode:priv_dir(reckon_nifs)only fires when the consumer has explicitly listed reckon_nifs as a dep. Plenty of consumers (including the gateway) hadn't. All three issues disappear when the NIFs live in the same package that uses them — which is how macula has been doing it all along. reckon-nifs 2.0.1 (the cleanup release shipped a few minutes before this one) is now the **final** release of that sidecar package. New consumers should depend only onreckon_db ~> 2.3; existing consumers pinned to reckon-nifs keep working because the wrappers retain the legacy lookup path. ### Other -rebar.config: packagelinksupdated from{"GitHub", ...}(which already pointed at codeberg.org but had a misleading label) to{"Codeberg", ...}. -rebar.config:pkg.filesextended to includenative/,priv/build-nifs.sh, the sixpriv/reckon_db.sobinaries, and theCONTRIBUTING.md+CODEOF_CONDUCT.mdfiles that landed in 2.2.2. -docs/dialyzer-backlog.md: the cleanup release that this document scheduled as v2.3.0 is bumped to **v2.4.0** since 2.3.0 is now this NIF-absorption release. ## [2.2.2] - 2026-05-17 ### Fixed — Normalize cluster status vocabulary Follow-up to 2.2.1'sreckon_db_clusterfacade.reckon_db_consistency_checkerusesconsensus/no_consensusin its result maps; the gateway'scluster_status/1converter expectshealthy/degraded/split_brain/no_quorumand falls through toCLUSTER_STATUS_DEGRADEDfor anything else. So a fully healthy 4-node cluster was being reported asDEGRADEDover gRPC despite consistency_checker correctly saying "consensus". The facade now translatesconsensus -> healthy,no_consensus -> split_brainbefore returning. ### Fixed — Documentation builds cleanrebar3 ex_docnow completes with zero warnings (was 24). Concrete fixes: -src/reckon_db_archive_backend.erl: dropped thereckon_db.hrlinclude (only the#event{}type alias was needed, not the record itself) and definedevent/0locally. Matches the pattern inreckon_db_log_backend.erl. -src/reckon_db_filters.erl: corrected five-specreturn types from the non-existentkhepri_evf:tree/0to the actual exported typekhepri_evf:tree_event_filter/0. -rebar.config(ex_docblock): addeddocs/genai-policy.mdto theextraslist so the README's link resolves on hexdocs, and addedskip_undefined_reference_warnings_onforCHANGELOG.md(the changelog legitimately references historic internal functions that are now private — those refs are documenting past fixes, not pointing at current API surface). ### Added —CONTRIBUTING.mdandCODE_OF_CONDUCT.mdCloses two gaps from the release checklist. CoC is the [Contributor Covenant 2.1](https://www.contributor-covenant.org/) verbatim. ### Documented — Dialyzer backlogrebar3 dialyzercurrently surfaces 182 warnings under the strict[underspecs, unmatched_returns, error_handling, unknown]configuration. None were introduced by 2.2.1 or 2.2.2 — these are latent issues inherited from earlier 2.x releases (the v2.2.0 already on hex carries the same count, minus 15 cleared as a side-effect of thereckon_db_clusterandreckon_db_subscriptions:subscribe/5work in 2.2.1/2.2.2). The full categorized backlog is at [docs/dialyzer-backlog.md](docs/dialyzer-backlog.md), with the raw warnings file atdocs/dialyzer-warnings-2.2.2.rawfor posterity. Clearing the backlog is scheduled as **v2.3.0**. ## [2.2.1] - 2026-05-17 ### Fixed — Add missingreckon_db_clusterfacadereckon_db_gateway_workerhad fourhandle_call/3clauses ({verify_cluster_consistency, },{quickhealth_check, },{verifymembership_consensus, },{checklog_consistency, }) that all called into areckondb_clustermodule which never existed — a dangling reference left over from theesdb → reckondbrename in v2.0.0. The bug was invisible until reckon-gateway 0.4.x exposed those RPCs over gRPC and a client (the new reckon-go SDK) actually called them — at which point they hung inreckongater_retry's exponential-backoff loop until the caller timed out, because each retry attempt died on{undef, [{reckon_db_cluster, ...}]}inside the gateway worker. This release addsreckon_db_clusteras a thin facade over [[reckon_db_consistency_checker]] andra_leaderboard: -health_check/1— cheap liveness check (quorum + leader presence). Used byHealthService.Check. -verify_consistency/1— full cluster consistency verdict (membership + leader consensus + quorum). Used byHealthService.VerifyClusterConsistency. -verify_membership/1— membership consensus across nodes. Used byHealthService.VerifyMembershipConsensus. -check_log_consistency/1— Raft log replication check. Used byHealthService.CheckRaftLogConsistency. The facade is intentionally stateless — it gathers state from ra/khepri on demand rather than depending on the (unsupervised)reckon_db_consistency_checkergen_server, so it works in bothsingleandclustermodes. ## [2.2.0] - 2026-05-17 ### Added — Cluster-wide store discovery + watcher APIreckon_db_store_registry' now provides genuine cluster-wide discovery for the EventStore-style "ephemeral store" model: stores exist when their supervision tree is running on at least one cluster node, and the registry tracks the union of who is currently announcing themselves. No CreateStore/DeleteStore — lifecycle stays a deployment concern. #### subscribe/1' +unsubscribe/1' (new) Public API for live store-topology events. Subscribed processes receive: {store_event, announced | retired, EntryMap} as stores come and go anywhere in the cluster. EntryMap matches the list_stores/0' shape. Subscribers are monitored — dead pids are pruned automatically via the registry'sDOWN' handler, so no explicit unsubscribe is needed when the watcher crashes. This is the substrate for the new gRPC reckon.gateway.v1.StoresService.WatchStores' RPC in reckon-gateway 0.4.0. ### Fixed — Cluster-wide discovery actually works Two latent bugs that meant each node only knew about its own local store, despite the cluster being healthy: 1. The previous version subscribed to a pg-mailbox message pattern that pg never emits ({pg, Scope, Group, {leave, , _}}'). Node-down cleanup was silently broken. 2. Announcement was one-way: when a registry came up, it broadcast its local store to peers but never asked peers for THEIR current state. A registry that booted after its peers had already announced ended up knowing only about itself. Fix: - Use pg:monitor/2', which returns CURRENT members and subscribes to live join/leave events idiomatically. The initial member list seeds a bilateral state-sync; later joins trigger a fresh state request to the new arrival. -peer_state_request' / peer_state_reply' cast pair — fully async, no try/catch around dead-peer calls, no timeouts. A dead peer just doesn't reply; merge proceeds with whatever arrived. Verified end-to-end against the 4-node beam cluster: every node sees all 4 store-instances afterpg:monitor' bootstrap. ### Other cleanups - find_entry/3' useslists:search/2', returns not_found' (was: alists:filter' returning false') - announce/unannounce handlers are clause-based on entry presence; the "no such entry" path is a no-op early return -notify_subscribers' uses plain Pid ! Msg' — runtime drops to dead pids silently, nocatch' wrapper needed ## [2.1.4] - 2026-05-17 ### Fixed — Cluster bootstrap robustness Four bugs in reckon_db_store_coordinator that, between them, could permanently strand a node outside the Raft cluster after a rough restart cycle: #### Infinite-timeout join khepri_cluster:join/2' internally useskhepri_app:get_default_timeout/0', which defaults to infinity'. Combined with the global lock acquired during the join, simultaneous- boot nodes could deadlock on lock contention forever. (Setting khepri'sdefault_timeout' app env globally would also affect every other khepri operation, so that's not a usable workaround.) The exported 2-arg version is now wrapped in a side process that's killed after ?KHEPRI_JOIN_TIMEOUT' (20s). On timeout the coordinator returnsfailed' and the retry-with-jitter timer picks up the next round. (khepri_cluster:join/3' is defined in the source but NOT exported in khepri 0.17.2 — onlyjoin/1' and join/2' are. Passing an explicit timeout via the 3-arg form fails with{undef, ...}'.) #### Self-clusters treated as active has_active_cluster/2 treated {ok, [SingleSelf]}' as an active cluster. Every freshly-booted Khepri node is a 1-member standalone cluster, so during a simultaneous boot every node saw every other node as a cluster and they all raced to join each other under the same global lock — the worst possible bootstrap shape. Tightened tolength(Members) > 1'. #### Coordinator election didn't drive cluster formation The original handle_no_existing_clusters just logged the elected coordinator and returned. Coordinator stayed as a 1-node cluster, non-coordinators sat in waiting' forever, nothing grew the cluster. With the self-clusters fix above, this previously-latent stalemate became reachable: 4 standalone clusters forever. Now: the elected coordinator stays as its 1-node cluster, but each non-coordinator actively joins via the coordinator. Once anyone joins, the coordinator's cluster has 2 members and subsequent retries from remaining non-coordinators find an active cluster via the regularhas_active_cluster' path. #### No retry on transient failure After waiting | failed | no_nodes', the coordinator gave up permanently. Added a jittered retry (3-8s) on the coordinator's own gen_server that re-attemptsdo_join_cluster/1' until status becomes joined'. #### Diagnostic for stale local state Beforekhepri_cluster:join' is called, verify the local Ra server is registered under the StoreId. If not, log a pointer to the `wipe-and-rejoin.sh' script in reckon-cluster-compose instead of hanging on infrastructure that never arrived. #### Verified end-to-end Cold-start torture against the 4-node beam cluster: Wipe all 4 data dirs, parallel docker compose up on all 4 All 4 nodes converge to 4-of-4 Raft membership without manual intervention Existing torture trio (leaderkill / partition_heal / subscription_failover) all pass against the freshly-formed cluster * Killing the new leader during the subscription scenario: new leader elected on the formerly-stuck beam00 node (proves it's a first-class member) ## [2.1.3] - 2026-05-17 ### Fixed — Cross-node subscription delivery + registration race Two bugs that, together, caused stream-scoped subscriptions to silently miss roughly half their events whenever the subscription was opened against a non-leader gateway. #### Cross-node delivery reckon_db_emitter:send_to_subscriber/4 had a single clause guarded on node(Pid) =:= node() plus a catch-all that returned ok. maybe_forward_events/2 had the same shape. When the Khepri trigger fired on the Raft leader and reckon_db_emitter_group:broadcast/3 picked an emitter that wasn't co-located with the subscriber pid, the emitter silently dropped the event. Each cluster node spins up its own emitter pool for every subscription (via reckon_db_leader_tracker and setup_event_notification), so the pg group typically holds 2+ emitters on different nodes — all carrying the same subscriber pid (the one captured by the client that called save_subscription). The random pick had a ~50% chance of landing on an emitter whose node didn't host the subscriber, and those events were lost. Pid ! Msg works fine across Erlang distribution; the local-only guard was the bug. Remote pids now receive via catch (Pid ! Msg). Liveness probing stays local-only because erlang:is_process_alive/1 is undefined for remote pids — the runtime's own dead-process semantics cover remote delivery to a dead pid. #### Registration race setup_event_notification registered the Khepri trigger BEFORE starting the emitter pool. Between those two steps, any event commit fired the trigger into an empty pg group — broadcast/3 logged "No emitters for ..." and dropped the event. Particularly noticeable on a hot stream during sub registration. Swapped the order to (persist names → start pool → register trigger), so the local emitter is in pg before the trigger goes live. #### Verification End-to-end on a 4-node cluster: subscriber received 25 events from our stream out of 25 writer-acked, contiguous version range 0..24, both pre- and post-leader-kill events delivered, zero cross-stream events received (the 2.1.2 catch-up filter still holds). ## [2.1.2] - 2026-05-17 ### Fixed — Catch-up filter Catch-up replay (the path that delivers historical events to a newly-registered subscription) ignored the subscription's selector and pushed the entire global event log to the subscriber, regardless of its declared filter. The Khepri trigger filter (live path) was correct; only the catch-up path was unfiltered. Net effect on an active store: every subscription opened with start_from = 0 received the full history of every stream, then flipped to correctly filtered live deliveries. Stream-scoped consumers had to discard 99%+ of what they received on attach. #### Implementation - New reckon_db_filters:matches/3 — in-memory predicate that evaluates a (Type, Selector) pair against an #event{} record. Handles by_stream (exact stream id; <<"$all">> matches all), by_event_type, and by_tags (set inclusion). by_event_pattern and by_event_payload pass through; live trigger filters them correctly so the gap is narrower. A real map-pattern evaluator is a follow-up. - do_catchup/5 now takes the subscription's type + selector and applies the predicate to each batch before sending. Read window through read_all_global still advances by raw batch size so the scan progresses even when nothing in a window matches. - deliver_catchup_batch separated into filtered/raw counts; logs "events scanned" rather than "delivered" so the metric reflects what catch-up actually saw. #### Behaviour change Subscribers that relied on receiving cross-stream events from a single by_stream subscription will now miss them. The intended contract is "subscribe per stream; use <<"$all">> for the global firehose" — this release makes the implementation match. ## [2.1.1] - 2026-05-15 ### Added — Backward-direction chain verification Closes the documented gap from 2.1.0. On integrity-enabled stores, reckon_db_streams:read/5,6 now verifies the chain for backward reads in exactly the same way as forward reads. The only behavioural difference between directions is the result-ordering of the returned events; the chain semantics are identical. #### Implementation The verifier walks events in forward order regardless of read direction (the chain runs forward through time and that's the direction it has to be checked in). For a backward read, the implementation reverses the result to forward order, runs the forward verifier, then reverses the verified list before returning so the caller still sees events highest-version-first. #### Behaviour change for callers Backward reads of integrity-enabled stores that previously succeeded against tampered storage now return `{error, {integrity_violation, }}. This is a hardening, not a regression: 2.1.0's behaviour was the documented gap. Callers relying on the old behaviour to access tampered data deliberately should use the existingOpts = #{verify => skipall}escape hatch. #### Testsbackward_read_bypasses_verification(which had asserted the gap) replaced with two tests inreckon_db_integrity_reads_SUITE: -backward_read_catches_tampering— symmetric assertion that the same tamper detected on forward reads is also detected on backward reads -backward_read_returns_events_in_descending_order— intact backward read returns[v4, v3, v2]with integrity fields populated Full regression: 514 eunit + 5/21/12/4 CT (writes/reads/snapshots/ subscriptions) = 556 tests pass. ## [2.1.0] - 2026-05-15 ### Added — Tamper-resistance for events and snapshots Implements Layers 2–5 of the cross-package design inplans/PLAN_TAMPER_RESISTANCE.md. Reckon-db now writes HMAC-protected, chain-hashed events when integrity is enabled on a store, and verifies them on every read surface. Requiresreckon_gater >= 2.1.0for the schema and verification primitives. #### Configuration#store_config{}gains anintegrityfield (defaultdisabled). To enable: ```erlang #store_config{ %% ... existing fields ... integrity = #{ enabled => true, key_source => {env_var, <<"RECKON_DB_KEY_MY_STORE">>} %% or: {sealed_file, "/path/to/key"} (mode 0600 required) } } ``` Keys are 32 random bytes (HMAC-SHA256). Loaded intopersistent_termat store startup; cleared on shutdown. Misconfiguration (missing env, bad base64, insecure file mode, wrong size) is fail-fast — the store refuses to start. #### Write path (Layer 2) -reckon_db_streams:append/4,5populatesprev_event_hash+macon every event when integrity is enabled. - New per-stream watermark stored under[metadata, integrity, chain_start, StreamId]. Set on the first integrity-bearing append to a stream. Events with version below the watermark stay legacy; events at or above must carry integrity fields. - Pre-existing legacy streams gain a watermark equal tocurrent_highest_version + 1on first integrity write — legacy data is preserved untouched. #### Read path (Layer 3) - Newreckon_db_streams:read/6accepts anOptsmap withverify => skip_legacy | strict | skip_all. Defaultskip_legacyfor backward compatibility. - Forward reads on integrity-enabled stores verify each event's MAC and chain link against a running tip. Failure surfaces as{error, {integrity_violation, }}— non-retriable, distinct fromwrongexpected_version. - Backward reads bypass chain verification in 2.1.0 (documented gap; MAC-only check possible in future). - New telemetry event[reckon, db, read, legacy_event_returned]fires when legacy events are returned underskip_legacy, for operator remediation tracking. #### Snapshot path (Layer 4) -reckon_db_snapshots:save/4,5populatesanchor_hash(chain hash of the event at the snapshot's version) +macwhen integrity is enabled. -load/2andload_at/3recompute the chain hash from the underlying event at load time and verify against the stored anchor. Detects post-snapshot stream tampering even when the snapshot itself is intact — the headline property this layer provides over MAC alone. - Save refused when no event exists at the target version or when the target event is legacy — a snapshot whose anchor cannot be established is unverifiable and worse than no snapshot. #### Subscription catch-up (Layer 5) -reckon_db_subscriptions:do_catchup/3MAC-verifies each integrity-bearing event before delivery. Cross-stream chain verification is intentionally NOT performed here (catch-up reads sort byepoch_usacross all streams; per-stream chain integrity belongs at the consumer / aggregate-rebuild layer). - Tampered event during catch-up halts replay and sends{subscription_error, {integrity_violation, }}to the subscriber. Emits[reckon, db, subscription, integrity, violation]telemetry. - Live events come from the write path with integrity fields already populated — no emitter-side change needed. #### New modules -reckondb_integrity_key— per-store HMAC key loader with validation (32-byte size, base64 decode, file mode 0600). -reckon_db_chain_watermark— per-stream watermark CRUD against the metadata tree. #### Tests 41 new Common Test cases plus 12 new eunit tests across four suites: -reckon_db_integrity_key_tests(12 eunit) -reckon_db_integrity_writes_SUITE(5 CT) -reckon_db_integrity_reads_SUITE(20 CT, 5 groups) -reckon_db_integrity_snapshots_SUITE(12 CT, 2 groups) -reckon_db_integrity_subscriptions_SUITE(4 CT) Full regression: 514 eunit + 41 integrity CT pass with zero existing-test regressions. ### Fixed -src/reckon_db_log_backend.erl— converted 11@doctags on-callbackdeclarations to plain%%comments. EDoc strict rules disallow@docbefore-callback; the previous shape brokerebar3 ex_docand would have blocked hex publication. Text content preserved verbatim. ### Changed -src/reckon_db.app.src—{links, [{"GitHub", ...}]}updated to{"Codeberg", ...}to match canonical hosting. -?RECKON_DB_VERSIONmacro ininclude/reckon_db.hrlsynchronised with the package version (was1.7.2, now2.1.0). -README.mdinstall snippet bumped from1.0.0to2.1.0. ### Out of scope (deferred) - Backward-direction read chain verification. - Cross-stream chain reconstruction on catch-up (per-event MAC only at that surface). - Ed25519 signatures for cross-trust-domain authenticity. Thesignaturefield is reserved on the schema but not populated; external authenticity is currently absent over the reckon-gateway wire. - Key rotation. Thekey_idslot is reserved ({1, MacBytes}shape); 2.1.0 always writeskey_id = 1. ## [2.0.0] - 2026-04-19 ### Changed **BREAKING**: Internal modules renamed fromesdbtoreckondbto match the overall reckon-db-org naming scheme. Most consumers go throughreckongater_apiand should not be affected directly, but any code that reaches into reckon-db internal modules must update: | Old module | New module | |---|---| |esdb_aggregate_nif|reckon_db_aggregate_nif| |esdb_archive_nif|reckon_db_archive_nif| |esdb_crypto_nif|reckon_db_crypto_nif| |esdb_filter_nif|reckon_db_filter_nif| |esdb_graph_nif|reckon_db_graph_nif| |esdb_hash_nif|reckon_db_hash_nif| |esdb_capability_verifier|reckon_db_capability_verifier| |esdb_revocation|reckon_db_revocation| ETS table atoms also renamed: -esdb_revoked_tokens→reckon_db_revoked_tokens-esdb_revoked_issuers→reckon_db_revoked_issuers### Dependencies - Bumpedreckon_gaterto~> 2.0(requires the corresponding renamed API from reckon-gater 2.0.0). - NIF binaries now loaded asreckon_dbnif.so— requires reckon-nifs 2.0.0. ### Migration Applications that go throughreckon_gater_apisee only the reckon-gater 2.0.0 renames. Direct-internal users: ```erlang %% Before {ok, Verified} = esdb_capability_verifier:verify(Token). %% After {ok, Verified} = reckon_db_capability_verifier:verify(Token). ``` Rebuild from clean:rm -rf _build rebar.lock && rebar3 compilewill re-fetch reckon_gater 2.0+ and reckon_nifs 2.0+ and recompile the renamed NIFs via the rustler hooks. ## [1.7.5] - 2026-03-22 ### Fixed - **Gateway worker version check bypass** —reckon_db_gateway_workerhad a duplicate version check (version_matches/2) that used atoms (any,stream_exists) instead of the integer constants (?ANY_VERSION = -2,?STREAM_EXISTS = -4) defined inesdb_gater_types.hrl. This causedappend_events/4via the gateway to rejectANY_VERSIONandSTREAM_EXISTSwith{wrong_expected_version, }. Removed the duplicate check — the gateway worker now delegates directly toreckondb_streams:append/4which handles all version constants correctly. --- ## [1.7.4] - 2026-03-22 ### Fixed - **Non-blocking nodeup handler** —handle_nodeup_cluster_joinnow runs entirely in a spawned process. Theshould_handle_nodeupcoordinator call was blocking the node monitor, causing 5s timeout crashes on every nodeup event (same pattern as the leader activation fix in 1.7.3). --- ## [1.7.3] - 2026-03-22 ### Fixed - **Non-blocking leader activation** —do_activatenow usesgen_server:castinstead of a blockinggen_server:callwith 10s timeout. When Khepri/Ra is still initializing,save_default_subscriptionsblocks on Khepri queries, causing the node monitor to time out and crash-loop every 15 seconds. The leader worker now handles activation asynchronously in its own process. --- ## [1.6.3] - 2026-03-19 ### Fixed - **Store Inspector**:list_streams/1returns[binary()]not[{binary(), integer()}]— all inspector functions were destructuring as tuples causing function_clause crashes ## [1.6.2] - 2026-03-19 ### Fixed - **Store Inspector**: Fixedbadargcrash insubscription_summary/1whensubscriber_pidis undefined - **Store Inspector**: Made snapshot listing defensive against per-stream errors - **Store Inspector**: Made subscription listing skip malformed entries instead of crashing - **Store Inspector**:format_pid/1handles undefined, binary, and non-pid terms gracefully ## [1.6.1] - 2026-03-19 ### Changed - Updated reckon_gater dependency to ~> 1.3.1 (includes inspector API exports) ## [1.6.0] - 2026-03-19 ### Added - **Store Inspector** (reckon_db_store_inspector): New module for aggregate store-level introspection. -store_stats/1— stream count, total events, snapshot count, subscription count -list_all_snapshots/1— all snapshots across all streams (summaries without data payloads) -list_subscriptions/1— all subscriptions with checkpoint positions -subscription_lag/2— events behind for a specific subscription -event_type_summary/1— census of event types with counts -stream_info/2— detailed info for a single stream (timestamps, snapshot coverage) - Gateway worker clauses for all inspector operations - Guide:guides/store_inspector.mdwith usage examples and performance notes - Architecture diagram:assets/store_inspector.svg## [1.5.1] - 2026-03-08 ### Added - **reckon_db_streams:has_events/1**: Check if a store contains at least one event. Reads 1 event viaread_all_global— correctly handles empty streams (truncation, GDPR erasure) unlike path-existence checks. Exposed via gateway worker. ## [1.5.0] - 2026-03-06 ### Added - **reckon_db_streams:read_all_global/3**: Read all events across all streams in global epoch_us order with offset/batch pagination. Used for catch-up subscriptions. ## [1.4.5] - 2026-03-06 ### Fixed - **Stale Khepri triggers after BEAM restart**: When a subscription already existed in Khepri (persisted from a previous run),reregister_subscriberonly updated the subscriber PID but did NOT re-register the Khepri trigger. The trigger's stored procedure (an Erlang fun/closure) becomes stale after a BEAM restart, so new events written to the store would never fire the notification mechanism. This caused subscription-based event delivery to silently stop working after daemon restarts. Fixed:reregister_subscribernow also re-creates the filter and re-registers the Khepri trigger, ensuring the stored procedure is fresh. ## [1.4.4] - 2026-03-06 ### Fixed - **Telemetry handler crash on subscription created**:handle_event(?SUBSCRIPTION_CREATED, ...)pattern-matched on#{subscription_id := }but the metadata fromsubscribe/5sendssubscriptionnameinstead. This caused abadmatchthat detached the telemetry logger handler for the entire session. Fixed: usemaps:get/3with fallback. ## [1.4.3] - 2026-03-06 ### Fixed - **Crash inupdate_subscriber_pidon re-subscribe**:reckon_db_subscriptions_store:get/2returnssubscription() | undefined, not{ok, subscription()} | {error, }. The re-registration code from v1.4.2 pattern-matched on{ok, Existing}which caused acaseclausecrash, killing the gateway worker and preventing all subscriptions from being set up on that store. Fixed: match on the record directly withis_recordguard. ## [1.4.2] - 2026-03-06 ### Fixed - **Subscriptions not re-registering subscriber PID after restart**: When a projection re-subscribes on startup, the subscription already exists in Khepri (persisted from the previous BEAM instance). Previously this returned{error, {already_exists, }}and the new subscriber PID was never registered. The emitter pool delivered events to the dead PID from the previous run, so projections never received events and read models stayed empty/stale after restart. Fix: when a subscription already exists and a newsubscriberpidis provided, update the stored subscription with the new PID and return{ok, Key}. ### Changed - **Eliminated all deep case/if nesting across codebase**: Refactored ~50 instances of depth-2+ nesting across 25 source files to max depth 1. Extracted helper functions, used pattern matching on function heads, and pipeline patterns. No behavioral changes. ## [1.4.1] - 2026-03-06 ### Fixed - **Subscription health monitor kills valid subscriptions after restart**: The health monitor treated subscriptions with deadsubscriber_pidas stale and deleted them, even when the emitter pool was running and actively serving events. After a daemon restart, ALL persisted subscriptions have dead PIDs (from the previous BEAM instance), so the health checker would kill every domain subscription ~2 minutes after boot. This left projections without event feeds and read models empty/stale. Fix: subscriptions with deadsubscriber_pidbut a running emitter pool are now treated as healthy (restarted subscription from a previous BEAM instance). - **App-level telemetry crashes handler on startup**:emit_start_telemetry()fired[reckon_db, store, started]with app-level metadata (#{application => reckon_db, version => ...}) instead of the expected#{store_id := ...}. This caused abadmatchinreckon_db_telemetry:handle_event/4, which detached the entire telemetry logger handler for the rest of the session. Removed the mistyped app-level telemetry events (per-store telemetry inreckon_db_storeis unaffected). - **StaleRECKON_DB_VERSIONmacro**: Updated from"0.1.0"to"1.4.1". ## [1.4.0] - 2026-03-06 ### Fixed - **Per-store Ra system isolation**: Each ReckonDB store now creates its own dedicated Ra system with separate WAL, segments, and DETS files. Previously, all stores shared the defaultkhepriRa system, causing all event data from every bounded context to be written into a single WAL file (whichever store started first owned the shared WAL directory). This affected both single and cluster modes. ## [1.3.3] - 2026-03-05 ### Fixed - **Late subscription event delivery**: Subscriptions registered after leader activation had Khepri triggers but no emitter workers, silently dropping events until the health monitor detected missing pools (up to 2 minutes).setup_event_notificationnow eagerly starts the emitter pool when the emitter supervisor is available, using pattern matching onwhereis/1to avoid agen_server:calldeadlock when called from within the leader worker during default subscription setup. ### Added -late_subscribe_starts_pool_immediatelyintegration test inreckon_db_emitter_autostart_SUITEverifying that the emitter pool exists immediately aftersubscribe/5returns when the leader is active. ### Changed - Bumpedreckon_gaterdependency to~> 1.1.3(includesdebug_infofor dialyzer) ## [1.3.2] - 2026-02-21 ### Fixed - **pg scope process dies silently**:pg:start_link(?RECKON_DB_PG_SCOPE)was called fromreckon_db_app:start/2, creating an unsupervised pg process linked only to the application master. When it died, no supervisor restarted it, silently breaking ALL event delivery (emitter workers join pg groups for subscription routing). Moved pg scope startup intoreckon_db_sup:init/1as the first supervised child withrestart => permanent, ensuring it is always restarted on failure. ### Added -reckon_db_pg_scope_SUITEintegration tests verifying pg scope supervision, automatic restart after crash, and full event delivery after scope restart. ## [1.3.0] - 2026-02-20 ### Fixed - **Leader detection in single mode**:reckon_db_node_monitorused a one-shot leader check in single mode that never rescheduled. If Ra leader election hadn't completed by the first check, the LeaderWorker never activated and emitter pools never started. Fixed to retry until leader is detected, then stop polling (no leadership changes in single-node mode). - **Node monitor placement**: Movedreckon_db_node_monitorfromcluster_sup(cluster mode only) tosystem_sup(all modes). The node monitor must run in single mode too to detect Ra leader and activate leader responsibilities. - **Supervisor strategies**: Changednotification_supandleader_supfromone_for_onetorest_for_one. Ifleader_supcrashes,emitter_supmust restart to prevent stale emitter pools running without leader coordination. Ifleader_trackercrashes,leader_workermust restart to re-establish dependency on tracking infrastructure. ### Added - **Subscription health monitor** (reckon_db_subscription_health): Periodic health checks (default 60s) that detect and clean up stale subscriptions (dead subscriber), orphaned emitter pools (pool without subscription), and missing emitter pools (subscription without pool). Only performs cleanup on the Ra leader node. Includes on-demandhealth_check/1API returning a health report map. - **Dead subscriber cleanup in emitter**: When an emitter worker detects its subscriber PID is dead during event delivery, it now asynchronously stops the emitter pool (matching ex-esdb'ssend_or_kill_poolpattern). Previously dead subscribers accumulated silently. - **Emitter autostart integration tests**: New CT suitereckon_db_emitter_autostart_SUITEwith 13 end-to-end tests covering leader activation, subscription lifecycle, event delivery, dead subscriber cleanup, and health monitor operation. ## [1.2.7] - 2026-02-18 ### Fixed - **Persistence worker crash on undefined options**:get_persistence_interval/1calledmaps:get/3on theoptionsfield ofstore_config, which crashed with{badmap, undefined}whenoptionswas not explicitly set. Fixed by adding a guard clause foris_map(Options)and a fallback clause that returns the default persistence interval. Also set the default value ofoptionsin thestore_configrecord to#{}(empty map) to prevent this class of bug in other code paths. ## [1.2.6] - 2026-02-13 ### Fixed - **Subscription id not populated**:subscribe/5created the#subscription{}record without setting theidfield, leaving it asundefined. The subscription key was computed and used for Khepri storage and trigger registration, but the subscription record passed tonotify_created(and thus to the leader_tracker and emitter pool) still hadid = undefined. This caused emitter workers to join pg group{StoreId, undefined, emitters}while Khepri triggers broadcast to{StoreId, CorrectKey, emitters}— a different group. Events were silently dropped because no emitters were found in the broadcast group. Fixed by settingSubscription#subscription{id = Key}before passing to downstream consumers. ## [1.2.5] - 2026-02-13 ### Fixed - **Stream subscription filter path mismatch**:by_stream/1was stripping the category prefix from stream IDs (e.g.,<<"test$delivery-001">>became<<"delivery-001">>), creating Khepri trigger filters that never matched stored events. This caused ALL stream-based subscriptions to silently fail — triggers never fired, subscribers never received events. Fixed to use the full stream ID in the filter path. - **Event type filter record matching**:by_event_type/1used a map pattern (#{event_type => Type}) to match stored events, but events are stored as#event{}records (tuples). Map patterns cannot match records. Fixed to use proper record pattern matching with#event{event_type = Type, = '_'}. ### Added - **Subscription delivery integration tests**: New CT suitereckon_db_subscription_delivery_SUITEwith 5 end-to-end tests verifying the full subscribe → append → trigger → emitter → deliver pipeline. ## [1.2.4] - 2026-02-13 ### Fixed - **Subscription Filter Error Handling**:create_filter/2errors no longer crash the gateway worker. Invalid stream names (e.g., missing$separator) now return{error, {invalid_filter, Reason}}instead of propagating tokhepri_evf:wrap/1which caused afunction_clausecrash. - **Gateway Worker Resilience**:handle_castforsave_subscriptionnow matches the result and logs a warning on failure instead of crashing. Previously, a single invalid subscription could crash the worker and lose all 28+ pending subscription messages in its queue. ## [1.2.3] - 2026-02-06 ### Fixed - **Subscription Filter Types**: Fixedcreate_filter/2function_clause error - Added support for gater-style subscription types:by_stream,by_event_type,by_event_pattern,by_event_payload,by_tags- Maintains backward compatibility with evoq-style types - Required for reckon_evoq_adapter type translation through the gater layer ## [1.2.2] - 2026-02-01 ### Documentation - **Event Envelope Documentation**: Improved event structure documentation - Added note about evoq event envelope inguides/event_sourcing.md- Documented metadata standardization (required vs optional fields) - Cross-referenced evoq Event Envelope Guide - Clarified simplified vs full envelope formats ## [1.2.1] - 2026-01-21 ### Fixed - **Documentation**: Corrected asset paths for hexdocs SVG rendering - Changed../assets/toassets/in all guides ## [1.2.0] - 2026-01-21 ### Added - **Distributed Store Registry**: Cluster-wide store discovery using pg groups -reckon_db_store_registryGenServer with pg-based distributed membership - Automatic store announcement/unannouncement on start/stop - Cross-node store visibility via broadcast mechanism -list_stores/0- List all stores in the cluster -get_store_info/1- Get detailed info about a specific store -list_stores_on_node/1- List stores on a specific node - 11 new unit tests for store registry - Gateway worker calls registry directly (no facade layer) ## [1.1.1] - 2026-01-21 ### Added - **Documentation**: Added Event Sourcing Paradigms guide to hexdocs - Entity-Centric (Traditional DDD) - Relationship-Centric (DCB - Dynamic Consistency Boundaries) - Process-Centric (Dossier metaphor with tags) ## [1.1.0] - 2026-01-21 ### Added - **Tag-Based Querying**: Cross-stream event queries using tags -read_by_tags/4- Query events by tags across all streams - Support forany(union) andall(intersection) matching modes - Tags field added to event records and storage - 15 new unit tests for tag filtering - Tags are for QUERY purposes only, NOT for concurrency control ### Changed - **Dependencies**: Updated reckon_gater from~> 1.0.3to~> 1.1.0for tags support ## [1.0.3] - 2026-01-19 ### Changed - **Dependencies**: Updated reckon_gater from exact1.0.0to~> 1.0.3to include critical double-wrapping bugfix ## [1.0.2] - 2026-01-09 ### Fixed - **Documentation**: Minor documentation improvements ## [1.0.0] - 2026-01-03 ### Changed - **Stable Release**: First stable release of reckon-db under reckon-db-org - All APIs considered stable and ready for production use - Updated Dockerfile with correct package names (reckon_db) - Fixed guide asset paths for hexdocs compatibility ## [0.4.6] - 2025-12-26 ### Fixed - **Dependency conflict**: Removed directradependency (khepri provides it). Updated toreckon_db_gater ~> 0.6.5which removed stale ra from its lock file. ## [0.4.5] - 2025-12-26 ### Fixed - **Dependency conflict**: Updatedradependency from exact2.16.12to~> 2.17.1to resolve conflict withreckon_db_gater ~> 0.6.4which requiresra ~> 2.17.1## [0.4.4] - 2025-12-22 ### Added - **Configuration Guide**: Comprehensive configuration documentation - Store configuration options (data_dir, mode, pool sizes) - Health probing configuration - Consistency checking and persistence intervals - Erlang (sys.config) and Elixir (config.exs) examples - Complete development/staging/production examples - Performance tuning recommendations - Telemetry events reference ## [0.4.3] - 2025-12-22 ### Added - **Gateway Worker Handlers**: -delete_stream- Delete streams via gateway -read_by_event_types- Native Khepri type filtering via gateway -get_subscription- Get subscription details including checkpoint These handlers support the erl-evoq-esdb adapter improvements. ## [0.4.2] - 2025-12-22 ### Added - **Cluster Consistency Checker** (reckon_db_consistency_checker.erl): - Split-brain detection via membership consensus verification - Leader consensus verification across all cluster nodes - Raft log consistency checks (term and commit index) - Quorum status monitoring with margin calculation - Four status levels:healthy,degraded,split_brain,no_quorum- Configurable check intervals (default: 5000ms) - Status change callbacks for alerting - Telemetry events:[reckon_db, consistency, ...]- **Active Health Prober** (reckon_db_health_prober.erl): - Fast failure detection via active probing (default: 2000ms intervals) - Three probe types:ping,rpc,khepri- Configurable failure threshold (default: 3 consecutive failures) - Node status tracking:healthy,suspect,failed,unknown- Recovery detection with callbacks - Telemetry events:[reckon_db, health, ...]- **Cluster Consistency Guide** (guides/cluster_consistency.md): - Split-brain problem explanation and prevention strategies - Consistency checker usage and configuration - Health prober integration patterns - Quorum management and recovery procedures - Circuit breaker and load balancer integration examples - **Architecture Diagrams** (SVG): -assets/consistency_checker.svg- Consistency checker architecture -assets/split_brain_detection.svg- Split-brain detection flow -assets/health_probing.svg- Health probing timeline ### Tests - 35 unit tests for consistency checker - 37 unit tests for health prober - All 72 new tests passing ## [0.4.1] - 2025-12-22 ### Added - **Server-Side Documentation Guides**: -guides/temporal_queries.md- Point-in-time queries, timestamp filtering, cluster behavior -guides/scavenging.md- Event lifecycle, archival backends, safety guarantees -guides/causation.md- Causation/correlation tracking, graph building, DOT export -guides/stream_links.md- Derived streams, filter/transform patterns -guides/schema_evolution.md- Schema registry, version-based upcasting, validation -guides/memory_pressure.md- Pressure levels, callbacks, integration patterns -guides/storage_internals.md- Khepri paths, version padding, cluster replication - **Architecture Diagrams** (SVG): -assets/temporal_query_flow.svg- Temporal query processing flow -assets/scavenge_lifecycle.svg- Event lifecycle state machine -assets/causation_graph.svg- Causation chain visualization -assets/stream_links.svg- Stream linking architecture -assets/schema_upcasting.svg- Schema version upcasting flow -assets/memory_levels.svg- Memory pressure level thresholds -assets/khepri_paths.svg- Khepri storage path structure ### Changed - **Documentation Improvements**: - Replaced ASCII diagrams with professional SVG graphics -snapshot_recovery.svg- Performance comparison visualization -event_fanout.svg- Multi-subscriber event delivery diagram - Updatedrebar.configex_doc with new guides organized into Core Concepts, Advanced Features, and Operations sections ## [0.4.0] - 2025-12-22 ### Added - **Enterprise Edition NIFs**: High-performance Rust NIFs with pure Erlang fallbacks - Community Edition (hex.pm) uses pure Erlang implementations - Enterprise Edition (git + Rust) gets 5-100x speedups for specific operations - Automatic fallback detection viapersistent_term- **reckon_db_crypto_nif** (Phase 1): -nif_base58_encode/1- Fast Base58 encoding for DIDs -nif_base58_decode/1- Fast Base58 decoding - Uses Bitcoin alphabet, ~5x faster than pure Erlang - **reckon_db_archive_nif** (Phase 2): -nif_compress/1,2- Zstd compression with configurable level -nif_decompress/1- Zstd decompression -nif_compress_batch/1,2- Batch compression for multiple items -nif_decompress_batch/1- Batch decompression - ~10x faster than zlib, better compression ratios - **reckon_db_hash_nif** (Phase 3): -nif_xxhash64/1,2- 64-bit xxHash with optional seed -nif_xxhash3/1- Modern xxHash3 (SIMD optimized) -nif_partition_hash/2- Hash to partition number -nif_stream_partition/3- Combined store+stream routing -nif_partition_hash_batch/2- Batch hashing for bulk ops -nif_fnv1a/1- FNV-1a for small keys -nif_fast_phash/2- Drop-in phash2 replacement - **reckon_db_aggregate_nif** (Phase 3): -nif_aggregate_events/2- Bulk fold with tagged value semantics -nif_sum_field/2- Vectorized sum accumulation for numeric fields -nif_count_where/3- Count events matching field condition -nif_merge_tagged_batch/1- Batch map merge with tagged values -nif_finalize/1- Unwrap tagged values ({sum, N}, {overwrite, V}) -nif_aggregation_stats/1- Event statistics (counts, unique fields) - **reckon_db_filter_nif** (Phase 3): -nif_filter_events/2- Filter events by compiled predicate -nif_filter_count/2- Count matching events without collecting -nif_compile_predicate/1- Pre-compile filter predicates -nif_partition_events/2- Partition events by predicate (matching/non-matching) -nif_first_match/2- Find first matching event -nif_find_all/2- Find all matching events with indexes -nif_any_match/2,nif_all_match/2- Boolean aggregate predicates - **reckon_db_graph_nif** (Phase 4): -nif_build_edges/1- Build edge list from event causation relationships -nif_find_roots/1,nif_find_leaves/1- Find root/leaf nodes -nif_topo_sort/1- Topological sort (Kahn's algorithm via petgraph) -nif_has_cycle/1- Detect cycles in causation graph -nif_graph_stats/1- Calculate node/edge/depth statistics -nif_to_dot/1,2- Generate Graphviz DOT format -nif_has_path/2- Check if path exists between nodes -nif_get_ancestors/2,nif_get_descendants/2- BFS path finding ### Changed - **Build profiles**: - Addedenterpriseprofile with Rust NIF compilation hooks - Addedenterprise_testprofile for testing with NIFs - Build withrebar3 as enterprise compileto enable NIFs ### Documentation - Updated README with Enterprise/Community edition information - Added NIF function documentation with academic references ## [0.3.1] - 2025-12-20 ### Changed - **Version padding**: Increased from 6 to 12 characters (?VERSION_PADDINGmacro) - Previous: 999,999 events per stream max (~2.7 hours at 100 events/sec) - Now: 999,999,999,999 events per stream max (~317 years at 100 events/sec) - Supports long-running neuroevolution, IoT, and continuous event streams ### Fixed - **EDoc errors**: Removed backticks and markdown from EDoc comments (breaks hex.pm docs) ## [0.3.0] - 2025-12-20 ### Added - **Capability-Based Security** (reckon_db_capability_verifier.erl,reckon_db_revocation.erl): - Server-side verification of UCAN-inspired capability tokens - Ed25519 signature verification using issuer's public key from DID - Token expiration and not-before time validation - Resource URI pattern matching (exact, wildcard suffix, prefix) - Action permission checking with wildcard support - Token revocation management (ETS-based, gossip integration planned) - Issuer revocation for compromised identities - Content-addressed token IDs (CIDs) for revocation tracking - Comprehensive unit tests (13 verifier tests + 6 revocation tests) This completes Phase 3 of the decentralized security implementation. Client-side token creation is in reckon-gater, server-side verification is here. ### Changed - **Documentation**: Replaced ASCII diagrams with SVG in README and guides ### Fixed - **README API documentation**: Fixed incorrect function signatures - Subscriptions: Added missingunsubscribe/3,get/2functions - Snapshots: Fixedload/3→load_at/3,delete/3→delete_at/3, addedexists/2,exists_at/3- Aggregator: Completely rewrote section - was showing non-existent API (foldl/4,foldl_from_snapshot/4) - **guides/snapshots.md**: Fixedload/3→load_at/3,delete/3→delete_at/3, rewrote aggregator example - **guides/cqrs.md**: Fixed subscription key usage in emitter group join - **guides/subscriptions.md**: Fixed invalid map access syntax - **guides/event_sourcing.md**: Fixed aggregator foldl signature (takes events list, not store/stream) ## [0.2.0] - 2024-12-19 ### Added - **End-to-end tests**: 24 comprehensive e2e tests for gater integration: - Worker registration (4 tests) - Stream operations via gater (9 tests) - Subscription operations (4 tests) - Snapshot operations (4 tests) - Load balancing (3 tests) - **Subscriptions**: Addedack/4function for acknowledging event delivery ### Fixed - **Gateway worker API compatibility**: -get_versionnow handles integer return correctly - Snapshot operations use correct function names (save,load_at,delete_at) - Subscription unsubscribe uses correct 3-arg version - **Header conflicts**: Addedifndefguards forDEFAULT_TIMEOUT` macro ### Changed - reckon-gater integration: Updated to work with gater's pg-based registry (replacing Ra) - *Test counts: Now 72 unit + 53 integration + 24 e2e = 149 total tests ## [0.1.0] - 2024-12-18 ### Added - Initial release of reckon-db, a BEAM-native Event Store built on Khepri/Ra - Event stream operations: - append/4,5 - Write events with optimistic concurrency control - read/5 - Read events from streams (forward/backward) - get_version/2 - Get current stream version - exists/2 - Check if stream exists - list_streams/1 - List all streams in store - delete/2 - Soft delete streams - Subscription system: - Stream subscriptions - events from specific streams - Event type subscriptions - events by type across streams - Pattern subscriptions - wildcard stream matching - Payload subscriptions - content-based filtering - Snapshot management: - save/5 - Save aggregate state snapshots - load/2,3 - Load latest or specific version snapshots - list/2 - List all snapshots for a stream - delete/3 - Delete old snapshots - Aggregation utilities: - foldl/4 - Fold over events with accumulator - foldl_from_snapshot/4 - Fold starting from latest snapshot - Cluster support: - UDP multicast discovery (LibCluster gossip compatible) - Automatic Khepri/Ra cluster formation - Node monitoring and failover - Leader election and tracking - Emitter pools for high-throughput event delivery - Partitioned writers for concurrent stream writes - BEAM telemetry integration with configurable handlers - Comprehensive test suite (72 unit + 53 integration tests) - Educational guides: - Event Sourcing fundamentals - CQRS patterns - Subscriptions usage - Snapshots optimization ### Dependencies - Khepri 0.17.2 - Raft-based distributed storage - Ra 2.16.12 - Raft consensus implementation - Telemetry 1.3.0 - BEAM telemetry for observability