barrel_vectordb_server (barrel_vectordb v2.1.2)
View SourcePer-store gen_batch_server managing RocksDB, vector index, and embeddings
Each store runs as a separate gen_batch_server registered under its name. Handles all document operations, search, and embedding coordination.
Uses gen_batch_server to automatically batch concurrent write operations into single atomic RocksDB WriteBatch operations, improving throughput under concurrent load.
Supports pluggable vector index backends: - hnsw: Pure Erlang HNSW implementation (default) - faiss: Facebook FAISS via NIF binding (optional, requires barrel_faiss)
Summary
Functions
Add document with auto-embedding.
Add multiple documents.
Index a vector without storing text/metadata (see barrel_vectordb).
Index multiple vectors without storing text/metadata.
Add document with explicit vector.
Add multiple documents with pre-computed vectors (bulk insert).
Trigger BM25 index compaction (disk backend only).
Get BM25 index information.
Checkpoint HNSW index to disk.
Get document count.
Delete document.
Embed single text.
Embed multiple texts.
Get embedder information.
Get document by ID.
The store's storage path (RocksDB dir; bm25/diskann nest under it).
Handle a batch of operations. Partitions operations into reads (processed immediately) and writes (batched atomically).
Peek at documents (sample without search).
Search with text query.
Search with BM25 text query. Opts: #{k => integer()}
Hybrid search combining BM25 and vector search. Opts: #{k => integer(), bm25_weight => float(), vector_weight => float(), fusion => rrf | linear}
Search with vector query.
Start a named store. Config options: - db_path: RocksDB storage path - dimension: Vector dimension - hnsw: HNSW index configuration - batch: gen_batch_server options (max_batch_size, min_batch_size)
Get store statistics.
Stop a store.
Update document metadata (re-embeds the text).
Insert or update document.
Types
Functions
Add document with auto-embedding.
-spec add_batch(store_ref(), [{binary(), binary(), map()}]) -> {ok, #{inserted := non_neg_integer()}} | {error, term()}.
Add multiple documents.
Index a vector without storing text/metadata (see barrel_vectordb).
-spec add_index_only_batch(store_ref(), [{binary(), binary(), [float()]}]) -> {ok, #{inserted := non_neg_integer()}} | {error, term()}.
Index multiple vectors without storing text/metadata.
Add document with explicit vector.
-spec add_vector_batch(store_ref(), [{binary(), binary(), map(), [float()]}]) -> {ok, #{inserted := non_neg_integer()}} | {error, term()}.
Add multiple documents with pre-computed vectors (bulk insert).
Trigger BM25 index compaction (disk backend only).
Get BM25 index information.
-spec checkpoint(store_ref()) -> ok.
Checkpoint HNSW index to disk.
-spec count(store_ref()) -> non_neg_integer().
Get document count.
Delete document.
Embed single text.
Embed multiple texts.
Get embedder information.
Get document by ID.
The store's storage path (RocksDB dir; bm25/diskann nest under it).
Handle a batch of operations. Partitions operations into reads (processed immediately) and writes (batched atomically).
-spec peek(store_ref(), pos_integer()) -> {ok, [map()]}.
Peek at documents (sample without search).
Search with text query.
Search with BM25 text query. Opts: #{k => integer()}
Hybrid search combining BM25 and vector search. Opts: #{k => integer(), bm25_weight => float(), vector_weight => float(), fusion => rrf | linear}
Search with vector query.
Start a named store. Config options: - db_path: RocksDB storage path - dimension: Vector dimension - hnsw: HNSW index configuration - batch: gen_batch_server options (max_batch_size, min_batch_size)
Default batch settings optimized for vector DB workloads: - min_batch_size: 4 (responsive for single inserts, batches concurrent ones) - max_batch_size: 256 (reasonable upper bound for memory/latency)
Get store statistics.
-spec stop(store_ref()) -> ok.
Stop a store.
Update document metadata (re-embeds the text).
Insert or update document.