Changelog

View Source

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[1.0.0] - 2026-07-10

First stable release. The barrel_docdb API is frozen: it will not break without a major version.

Changed (breaking)

  • barrel_docdb:query/2,3 rejects a table-function source with {error, {table_fn_requires_barrel, Fn}}. The atom was table_fn_requires_facade; it was never in a published release.

Changed

  • Documentation refresh: the doc set now describes the HLC version-vector model (dropping the removed rev-tree functions put_rev/revsdiff), and the HTTP API pages point to barrel_server since barrel_docdb is embedded-only. See the umbrella CHANGELOG.

[0.8.0] - 2026-06-14

Removed (breaking)

  • The HTTP server and its REST/SSE/admin API are gone. barreldocdb is now embedded-only: use it as an Erlang library via the barrel_docdb module. This drops the livery dependency, the API-key store, the signed-peer registry, and the HTTP replication transport. Replicate in-process with barrel_rep_transport_local or your own barrel_rep_transport behaviour (URL replication targets now raise `{http_transport_removed, }`).

Fixed

  • find/2,3: an AND of two or more {compare, ...} conditions no longer returns the empty set.
  • find/2,3: {prefix, Path, Prefix} works as a sole condition.
  • Documents re-created over a tombstone are re-indexed, so find returns them.

Added

  • find/2,3 id scans: #{id_prefix => P} and #{id_range => {Start, End}} ordered primary-key range scans over the document id.
  • find/2,3 flat => true returns flat documents (Doc#{<<"id">>}) instead of the #{<<"id">>, <<"doc">>} wrapper.

Changed

  • OTP 29: dropped the deprecated prefix-catch operator reliance; added barrel_lib with reusable safe_* helpers.
  • Bumped rocksdb to 3.0.0 and instrument to 1.1.4.

Notes

  • Top-level fields whose key begins with _ are reserved metadata: they are not stored or indexed, so they cannot be queried.

[0.7.7] - 2026-06-11

Changed (breaking)

  • /_replicate now refuses to call out to anything not in the new peer registry. The request body must carry either peer_id (a registered peer's id, URL pulled from the registry) or a target URL that matches a registered peer's canonical URL. Anything else is rejected with 403 unregistered_peer. Closes the SSRF surface (security-review #6): an authenticated key holder can no longer aim outbound replication at 127.0.0.1, 169.254.169.254, RFC1918, or any unregistered host.
  • The inbound replication-receiving endpoints (_revsdiff, _put_rev, _sync_hlc) now require a valid Ed25519 signature from a registered peer (X-Peer-Id, X-Peer-Timestamp, X-Peer-Signature headers) in addition to the existing API key. Missing or invalid signatures return 401 peer_signature_required / unregistered_peer / invalid_peer_signature.
  • Outbound replication calls now always include peer_auth => true, i.e. every request between registered peers is signed.

Added

  • New module barrel_peer_registry (DETS-backed, file <data_dir>/peers.dets). Schema: peer_id, name, canonical url, public_key, databases, created_at, last_used.
  • Admin HTTP routes:
    • GET /peers — list registered peers.
    • POST /peers — register a peer. Body: {"name": "...", "url": "...", "public_key": "<base64>", "peer_id": "..."} for an explicit registration, or {"name": "...", "url": "...", "discover": true} to TOFU-fetch the remote's /.well-known/barrel pubkey.
    • GET /peers/:peer_id — fetch one entry.
    • DELETE /peers/:peer_id — revoke. All four routes require an admin API key.
  • Config knob replication_require_registered_peer (default true), BARREL_REPLICATION_REQUIRE_REGISTERED_PEER. Setting it to false re-enables the legacy free-form target URL behaviour as a migration escape hatch — scheduled for removal in 0.8.0.

Migration

  • Existing replication clients that call /_replicate with a raw target URL must register the target first (POST /peers {..., "discover": true} is the easy path on trusted networks), then either continue calling /_replicate with the same URL or switch to peer_id.
  • Inbound endpoints will start returning 401 to any caller that isn't a registered peer. Make sure both ends of every replication link register each other before deploying this version.

[0.7.6] - 2026-06-05

Changed (breaking)

  • Query timeout now hard-fails instead of silently truncating. When the document-fetch pipeline (or the barrel_parallel worker pool) does not finish within query_timeout_ms (default 30000ms, override via BARREL_QUERY_TIMEOUT_MS), the call raises error({query_timeout, Info}). The HTTP /_find endpoint maps this to 504 Gateway Timeout with a JSON/CBOR body of {"error":"query_timeout","completed":N,"total":T, "missing_batches":M,"timeout_ms":Ms}. Callers that previously received a partial {ok, Results, Meta} with no signal must now handle the 504.

Added

  • barrel_query_timeouts counter (Prometheus / OTel) records every pipeline or pool deadline expiry.
  • Configurable global deadline query_timeout_ms shared by barrel_query and barrel_parallel. Replaces the previous hardcoded 30s and 60s splits.

Fixed

  • barrel_parallel:pmap/2,3 spawn-fallback path used a bare receive with no deadline, so a stuck worker would hang the caller forever. Now bounded by the same query_timeout_ms.

[0.7.5] - 2026-06-04

Added

  • HTTP request body size cap and in-flight request cap via livery's middleware stack. Buffered bodies above http_max_body_bytes (default 4 MiB) return 413; concurrent requests above http_max_in_flight (default 1000) shed with 503. Both tunable via BARREL_HTTP_MAX_BODY_BYTES / BARREL_HTTP_MAX_IN_FLIGHT. Streamed attachment uploads bypass the body cap by design.

Changed

  • barrel_http_api_keys now reads the canonical data_dir config key, with a deprecation-warning fallback to legacy data_path. The fallback is scheduled for removal in 0.8.0.

Removed

  • Dead cowboy-era config keys http_acceptors, http_max_connections, and http_protocols. Livery's defaults (60s request timeout, 300s idle timeout) replace them; per-request caps are enforced by middleware.

[0.7.4] - 2026-06-04

Changed

  • The OpenAPI 3.1 document is now generated at boot from the route table in barrel_http_server and served live at GET /openapi.json. A Redoc UI is at GET /docs. Both are public (no API key required) so SDK generators and importers can pull the spec without credentials.
  • Per-route OpenAPI metadata (operation_id, summary, tags, optional parameters / request_body / responses) lives next to each route in barrel_http_server:routes/0. To update the doc, edit the route metadata; the next service start picks it up.

Removed

  • docs/api/openapi.yaml (hand-maintained spec) and the stale openapi.yaml at the repo root (0.5.0, referenced the JWT scheme we removed in 0.6.2). Replaced by the live endpoint.
  • docs/api/openapi.md rewritten to point at /openapi.json and /docs instead of the deleted YAML file.

[0.7.3] - 2026-06-04

Changed

  • Authentication and authorisation moved into a livery middleware. New module barrel_docdb_auth is a service-level middleware (one entry in barrel_http_server's middleware stack) that classifies every request by (path, method) into public / admin / {data, DbName, ReadOrWrite}, runs the API-key check, and 401/403s before the handler ever runs. Handlers no longer call maybe_authenticate / authorize / required_perm — those lived inside barrel_http_handler and have been removed.
  • Routes stay 3-tuples; there's no per-route auth decoration. The middleware parses the database name and required permission from the raw path (service-level middleware runs before livery's router sets path bindings, so we can't rely on livery_req:binding/2 there).
  • barrel_http_api_keys (DETS-backed key store) and barrel_peer_auth (replication Ed25519 signing) are unchanged. The validated key map is attached to the request via livery_req:set_meta(auth_ctx, _, _) for handlers that want to inspect the caller.

[0.7.2] - 2026-06-04

Changed

  • HTTP observability moves into livery middleware. barrel_http_server now wires livery_request_id, livery_instrument_trace, livery_instrument_metrics, and livery_access_log into the service stack. Every request gets a W3C traceparent propagation, an OpenTelemetry server span with semantic-convention attributes, the standard http.server.active_requests + http.server.request.duration metrics, and an access log line.
  • /metrics is now served by livery_metrics:handler/0. The Prometheus exposition format is unchanged; both livery's HTTP middleware and barrel_metrics's domain registrations feed the same instrument registry.
  • barrel_http_handler:handle/2 no longer wraps requests in barrel_trace:with_extracted_context/2 + with_http_span/3 — the middleware does it.

Removed

  • HTTP-shaped entries from barrel_metrics: barrel_http_requests (counter), barrel_http_request_duration_seconds (histogram), and the inc_http_requests/3 + observe_http_latency/3 helpers. Replaced by livery's OTel-semantic-convention names (http.server.active_requests, http.server.request.duration). Dashboards or alerts referencing the old names need updating.

Kept

  • Domain metrics (doc ops, query ops, replication, db gauges) and the barrel_trace:* helpers — they're used by domain code, not HTTP plumbing.

[0.7.1] - 2026-06-04

Changed

  • Switch livery from a git pin against main to the hex package 0.2.0.
  • Replace hackney with livery_client for outbound replication HTTP. barrel_rep_transport_http now uses livery_client:request/4 for the _changes/_put_rev/ _revsdiff/_sync_hlc calls, giving outbound replication the same composable middleware stack the server side has. The underlying transport remains hackney (via livery's livery_client_hackney adapter), but is no longer a direct barrel dep.
  • Drop hackney from rebar.config deps, barrel_docdb.app.src applications, and the relx release list. It still arrives as a transitive dependency of livery.

[0.7.0] - 2026-06-04

Changed

  • HTTP layer migrated from cowboy to livery. All HTTP endpoints, headers, and response shapes are unchanged; the framework swap is internal. The dep graph drops cowboy / cowlib / ranch and adds livery (and its transitive h1 / h2 / quic / ws / webtransport / barrel_mcp). Consumers building a release from source need livery in their build environment.
  • HTTP/2 is supported out of the box (livery serves H1 and H2 from one listener via h2c upgrade or prior knowledge). HTTP/3 is available via livery and can be enabled in a follow-up.
  • HTTP modules are now grouped under src/web/ (rebar3 src_dirs extended).
  • instrument is now consumed from hex (1.1.3) instead of the git pin, matching livery.

Internal

  • barrel_http_server calls livery:start_service/1 + livery_router:compile/1 instead of cowboy's listener / dispatch APIs.
  • barrel_http_handler returns value-based responses via livery_resp:new/3, livery_resp:stream/3. Per-action handle_action/3 clauses keep the same internal tuple shape; conversion is centralized in handle/2.
  • barrel_http_changes_stream uses livery_resp:sse/3 with a producer closure that owns the poll/heartbeat loop.

[0.6.4] - 2026-06-02

Security

  • API key permissions are now enforced. Before 0.6.4 the permissions field on an #api_key record was stored and returned but discarded by the HTTP handler, so a key declared as [<<"read">>] could still write, delete, replicate, or bulk-update. A new authorize/3 step in barrel_http_handler maps each (action, method) pair to a required permission (read / write / admin) and returns 403 when missing. is_admin = true and the "no keys configured yet" mode short-circuit. Admin-gated endpoints (/keys/*, /admin/*) are unchanged.
  • Chunked attachment metadata no longer goes through binary_to_term. The writer now stores chunked metadata as <<"BARREL_CHUNK_V1:", json:encode(Meta)/binary>>; the reader pattern-matches on the tag and falls back to the legacy term_to_binary blob via binary_to_term/2 with [safe] for one release (with a warning log). Removes a path that decoded user-controlled bytes as Erlang terms.
  • Query-op whitelist. convert_op/1 in the HTTP handler used to call binary_to_atom/1 on any unrecognized binary, which could exhaust the atom table. The function now accepts only the documented set (eq/==, ne/!=/=/=, gt/>, gte/>=, lt/<, lte/<=/=<) and rejects everything else with HTTP 400.

Deprecated

  • Legacy term_to_binary chunked-attachment metadata. Attachments created on 0.6.3 or earlier are still readable in 0.6.4 with a warning; the fallback will be removed in 0.7.0.

[0.6.3] - 2026-06-02

Security

  • MVCC: stale or missing _rev on update is now rejected with {error, conflict} (HTTP 409). Previously, barrel_docdb:put_doc/2,3 and the bulk path silently overwrote the stored body and reported a different winning rev — a classic lost-update bug. The replication path (put_rev, bulk entries carrying explicit history) is unchanged: the revtree merge handles conflict detection there.
  • Database names are validated. barrel_docdb:create_db/1,2 and delete_db/1 reject anything that does not match [a-z0-9_-]{1,63} (system databases keep their leading _). HTTP 400 is returned for invalid names at PUT /db/:db and DELETE /db/:db. Prevents filesystem traversal via the data directory.
  • delete_db/1 no longer shells out. Replaces os:cmd("rm -rf " ++ DbPath) with file:del_dir_r/1. Removes the shell-injection surface that depended on whatever ended up in DbPath.

[0.6.2] - 2026-06-02

Removed

  • JWT authentication (barrel_docdb_jwt, console_public_key config, bdb_ token branches in the HTTP handler) and the jose dependency. API key auth (ak_*) is the only bearer-token scheme.
  • Dead VDB benchmark code (bench/src/workloads/barrel_bench_vdb.erl, barrel_bench:run_vdb*, VDB portions of the HTTP bench) referencing the long-removed barrel_vdb module.
  • Stale test/docker/CHECKPOINT.md referencing removed barrel_rep_policy work.

Changed

  • Bumped instrument to v1.1.3 for Erlang/OTP 29 compatibility (replaces deprecated catch Expr with try ... catch).
  • Replaced deprecated catch Expr patterns in barrel_docdb, barrel_db_server, barrel_query, barrel_query_cursor, and the affected test suites; added barrel_store_rocksdb:safe_release_snapshot/1 helper.
  • Updated docs to drop references to removed modules (run_vdb benchmark; "Sharding Strategy" renamed to "Bucketing Strategy" for the time-bucketed posting list).

[0.6.1] - 2026-06-01

Changed

  • Moved to Erlang/OTP 28 (CI containers, Dockerfile builder, and docs).

Fixed

  • Docker image now boots on OTP 28: the runtime stage uses debian:trixie-slim to match the erlang:28 builder's glibc (it was left on bookworm-slim, so ERTS failed with GLIBC_2.38 not found).
  • Release CI job builds in the erlang container instead of installing Erlang from packages.erlang-solutions.com, which had failed the v0.6.0 release.
  • The relx release version was hardcoded 0.4.2; it now matches the application version.

[0.6.0] - 2026-05-26

Added

  • GET /.well-known/barrel node identity endpoint returning node_id, version, and (when peer auth is initialized) public_key, with no dependency on discovery or federation
  • Public barrel_docdb:node_id/0 seam (persistent _node_id system document) for an external discovery or cluster layer

Removed

  • Virtual Database (VDB) and sharding: removed barrel_vdb*, barrel_shard_map, and barrel_shard_rebalance with the /vdb HTTP endpoints
  • Federation and peer discovery: removed barrel_federation and barrel_discovery with the /_federation, /_peers, and /.well-known/barrel endpoints
  • Replication policies: removed barrel_rep_policy (chain/group/fanout/tiered patterns) and the /_policies endpoints; the replication engine (barrel_rep, barrel_rep_tasks, transports) is unchanged
  • Tiered storage: removed barrel_tier and the /db/:db/_tier/* endpoints; the created_at/expires_at/tier entity columns are kept as reserved fields so the on-disk format is unchanged
  • Materialized views: removed barrel_view, barrel_view_index, barrel_view_sup, the register_view/unregister_view/query_view/list_views/refresh_view API, and the /db/:db/_views/* endpoints

Changed

  • Reduced scope to a document-database core: CRUD with MVCC, declarative queries, changes feed, attachments, pub/sub, and the replication engine. Sharding, tiering, and clustering can be built on top via the public API (changes feed, revision primitives, HLC, system and local documents, store/index introspection, and metrics).
  • barrel_peer_auth is retained (used by the HTTP replication transport); its node id now persists in a _node_id system document.
  • Upgraded the instrument dependency to v1.1.2.

Fixed

  • Metrics exemplar reservoir table is no longer lost on an application stop/start. It is initialized in barrel_metrics:setup/0, which previously caused barrel_rep_tasks to crash on restart when the first histogram value was recorded.

[0.5.0] - 2026-04-04

Added

  • Views with Map/Reduce: Secondary indexes with map/reduce support
    • Module-based views with map/1 and optional reduce/3 callbacks
    • Query-based views with declarative key/value extraction
    • Built-in reduce functions: _count, _sum, _stats
    • Rereduce support for merging results from sharded queries via merge_reduced_results/2
    • Manual and automatic refresh modes
  • OpenAPI 3.0 Specification: Full API documentation with Swagger UI at /api-docs
  • barrel_docdb:fold_docs/4 with options support (limit, skip, start_key, end_key)
  • barrel_query:matches/2 for simple condition matching outside of queries

Changed

  • Migrated from legacy {Epoch, Counter} sequence format to HLC timestamps throughout
  • Removed barrel_sequence module (functionality consolidated in barrel_hlc)
  • Upgraded hlc to 3.0.3 and match_trie to 1.0.0
  • Upgraded rocksdb to 2.6.2
  • Removed unused bitmap dependency

Fixed

  • Multiple dialyzer warnings across modules
  • Dead code removal in fold_range functions
  • Unmatched return values in shard rebalancing

[0.4.1] - 2026-03-08

Fixed

  • Snapshot handle leak in chunked exists/prefix query paths where temporary snapshots were never released
  • Streaming attachment uploads now clean up orphaned chunks on failure via new abort_stream/1 function

Added

  • abort_attachment_writer/1 API to clean up partial attachment uploads
  • Snapshot support for pure compare queries ensuring read consistency
  • fold_compare_docids_with_snapshot/8 in barrel_ars_index
  • fold_range_posting_compare_with_snapshot/6 in barrel_store_rocksdb

[0.4.0] - 2026-03-05

Added

  • Authentication support for federation queries with bearer_token and basic_auth formats
  • Federation-level auth stored with config, per-query auth override in find options
  • doc_ids filtering for changes feed to subscribe to specific documents
  • Query-based filtering for changes feed with where conditions
  • GitHub Actions CI/CD workflows for automated testing and releases

Fixed

  • VDB replication setup by adding sync option to peer registration
  • JSON encoding of peer public keys in HTTP API responses

Changed

  • Upgraded hackney to 3.2.1

[0.3.2] - 2026-02-13

Fixed

  • SSE changes stream now correctly includes document bodies when include_docs=true
  • Fixed get_changes_full_scan to route to filtered path when include_docs is requested
  • Fixed get_changes_filtered to set NeedsDoc flag when include_docs=true

Added

  • changes_stream_include_docs test to verify document bodies in SSE events

[0.3.1] - 2026-02-13

Verified

  • SSE changes stream stability confirmed with all tests passing
  • since=now parameter handling working correctly
  • Heartbeat mechanism keeping connections alive

[0.3.0] - 2026-02-12

Added

JWT Authentication

  • ES256 (ECDSA P-256) JWT token validation for API authentication
  • Token format: bdb_<base64-encoded-JWT> prefix for barrel_docdb tokens
  • Required claims validation: sub, typ, oid, prm, exp
  • Workspace isolation via optional wid claim
  • Permission-based access control with is_admin flag
  • File-based or inline PEM key configuration

Usage Reporting

  • New barrel_docdb_usage module for database statistics
  • GET /admin/usage - Get usage stats for all databases
  • GET /admin/databases/:db/usage - Get stats for specific database
  • Stats include: document_count, storage_bytes, memtable_size, sst_files_size

Ed25519 Peer Authentication for P2P Replication

  • barrel_peer_auth gen_server for Ed25519 key management
  • Automatic keypair generation on startup
  • Request signing with canonical format: timestamp|peer_id|method|path|body_hash
  • 5-minute timestamp window for replay protection
  • HTTP headers: X-Peer-Id, X-Peer-Timestamp, X-Peer-Signature
  • Public key exposed via /.well-known/barrel discovery endpoint
  • Optional peer_auth option in HTTP transport (disabled by default)

Fixed

SSE Changes Stream Reliability

  • Reduced heartbeat interval from 60s to 30s (below Cowboy's idle_timeout)
  • Added idle_timeout => 120000 to Cowboy protocol options
  • Added request_timeout => infinity for long-running SSE streams
  • Fixed since=now parameter crash (was calling barrel_hlc:encode(now))
  • Simplified parse_since/1 to only accept valid base64-encoded HLC binaries

API Compatibility

  • Fixed barrel_docdb_usage to use db_pid/1 + get_store_ref/1 API
  • Updated all HTTP tests for hackney 3.x API (body in 4th tuple element)

Changed

  • Upgraded hackney to 3.0.2
  • Default heartbeat for SSE streams changed from 60s to 30s

Tests Added

  • barrel_docdb_usage_SUITE - 6 tests for usage statistics module
  • barrel_docdb_jwt_SUITE - 10 tests for JWT authentication
  • barrel_peer_auth_tests - 11 EUnit tests for peer authentication
  • HTTP usage endpoint tests (4 tests in barrel_http_SUITE)
  • SSE stream tests: changes_stream_since_now, changes_stream_heartbeat

[0.2.0] - 2025-01-12

Added

Virtual Databases (VDB) - Automatic Sharding

  • New VDB layer for horizontal scalability with automatic document sharding
  • Consistent hashing of document IDs for even distribution across shards
  • Scatter-gather queries across all shards with merged results
  • HTTP API endpoints for VDB operations (/vdb/:vdb/*)
  • Shard split and merge operations for rebalancing
  • Multi-datacenter sharding with zone-aware placement
  • Cross-node VDB configuration synchronization
  • VDB shard replication with automatic failover
  • Import from regular database to VDB (POST /vdb/:vdb/_import)
  • VDB benchmarks and comprehensive documentation

HTTP/2 Support

  • HTTP/2 cleartext (h2c) via HTTP Upgrade mechanism
  • HTTP/2 over TLS (h2) with ALPN negotiation
  • Graceful degradation to HTTP/1.1 for legacy clients
  • Environment variable configuration for TLS/HTTP2 in releases

Docker & CI

  • Multi-architecture Docker builds (amd64 + arm64)
  • GitLab CI pipeline with buildx for container builds
  • Docker multi-region VDB test suite

Documentation

  • VDB overview and sharding guide
  • Multi-datacenter deployment documentation
  • Advanced features guide with curl examples
  • HTTP API reference for VDB endpoints

Fixed

  • HTTP test suite isolation by unlinking server process
  • VDB cross-node sync authentication
  • Replication policy auth and checkpoint error handling
  • HTTP benchmarks and documentation links

Changed

  • Upgraded hackney to 2.0.0-beta.1
  • Added LZ4 and Snappy libraries for RocksDB native build

[0.1.0] - 2024-12-01

Added

  • Initial release
  • Document CRUD with MVCC revision trees
  • Declarative queries with automatic path indexing
  • Real-time subscriptions via MQTT-style path patterns
  • HTTP API with REST endpoints
  • Peer-to-peer replication with configurable patterns (chain, group, fanout)
  • Federated queries across multiple databases
  • Tiered storage with automatic TTL/capacity-based migration
  • Prometheus metrics for monitoring
  • HLC ordering for distributed event coordination