barrel_outbox (barrel_docdb v1.0.0)
View SourceGeneric tagged outbox: a durable work queue written atomically with the document batch.
A write carrying the put option outbox => [Tag] appends one entry per tag to the same RocksDB WriteBatch as the document, so the entry and the doc commit or fail together. Entries are keyed prefix | db | tag | hlc (the write's change HLC), giving time-ordered scans per tag and exact-key acknowledgements.
Rewrites of a tagged document replace the entry (the old HLC key is deleted in the same batch, mirroring the changes feed), so the outbox holds at most one pending entry per document per tag. Consumers fold a tag (caller-side, no writer round-trip), process the CURRENT state of each document, and ack by exact HLC key: if the document was rewritten mid-processing, the new entry lives at a different key and re-drives the consumer, so acking the processed key never loses work. Un-acked entries are the consumer's checkpoint; there is nothing else to persist.
docdb knows nothing about what tags mean; producers (for example the barrel's embedding indexer) define the semantics.
Summary
Functions
Decode an outbox entry value (without the key-derived hlc).
Encode an outbox entry value (compact binary, same style as the change entry codec).
Fold over the pending entries of a tag in HLC order. Runs in the caller's process (pure RocksDB range scan). Fun receives a decoded entry() and the accumulator and returns {ok, Acc} or {stop, Acc}. Options: limit (max entries to visit).
Fold over the pending entries of a tag with options.
Batch ops for a tagged write: one put per tag at the new HLC, plus a replace-delete of the previous entry when the doc existed. Deleting a key that was never written is a RocksDB no-op, so a previously untagged write is harmless.
Types
-type att_info() :: #{name := binary(), content_type := binary(), length := non_neg_integer(), digest := binary(), chunked => boolean(), chunk_size => pos_integer(), chunk_count => pos_integer()}.
-type change() :: map().
-type db_name() :: binary().
-type docid() :: binary().
-type entry() :: #{hlc := barrel_hlc:timestamp(), id := docid(), rev := revid(), deleted := boolean()}.
-type revid() :: binary().
-type seq() :: barrel_hlc:timestamp().
-type seq_string() :: binary().
-type tag() :: binary().
-type view_name() :: binary().
Functions
Decode an outbox entry value (without the key-derived hlc).
Encode an outbox entry value (compact binary, same style as the change entry codec).
-spec fold(barrel_store_rocksdb:db_ref(), db_name(), tag(), fun((entry(), term()) -> {ok, term()} | {stop, term()}), term()) -> term().
Fold over the pending entries of a tag in HLC order. Runs in the caller's process (pure RocksDB range scan). Fun receives a decoded entry() and the accumulator and returns {ok, Acc} or {stop, Acc}. Options: limit (max entries to visit).
-spec fold(barrel_store_rocksdb:db_ref(), db_name(), tag(), fun((entry(), term()) -> {ok, term()} | {stop, term()}), term(), map()) -> term().
Fold over the pending entries of a tag with options.
-spec write_ops(db_name(), [tag()], barrel_hlc:timestamp(), barrel_hlc:timestamp() | undefined, docid(), revid(), boolean()) -> [term()].
Batch ops for a tagged write: one put per tag at the new HLC, plus a replace-delete of the previous entry when the doc existed. Deleting a key that was never written is a RocksDB no-op, so a previously untagged write is harmless.