RuvectorElixir

Official Elixir bindings for ruvector, an ultra-fast embedded vector database and GraphRAG metadata engine written in Rust.

RuvectorElixir allows you to store, index, and query high-dimensional vector embeddings with sub-millisecond approximate nearest neighbor (ANN) search directly from BEAM applications—without external services, daemon processes, or network overhead.

Features

• Embedded & Zero-Dependency: Embedded vector storage with zero external database dependencies. Runs in-process inside your BEAM node.
• High-Performance NIF: Built with Rustler. Heavy computation and disk I/O are safely executed on Erlang dirty schedulers (DirtyCpu and DirtyIo) to prevent scheduler starvation.
• Multiple Distance Metrics:
  • :cosine (Cosine Similarity)
  • :euclidean (Euclidean / L2 Distance)
  • :dot_product (Dot Product)
  • :manhattan (Manhattan / L1 Distance)
• HNSW & Flat Indexing: Graph-based Hierarchical Navigable Small World (HNSW) indexing for sub-millisecond retrieval on large datasets, plus exact flat indexing.
• Rich JSON Metadata Filtering: Filter vector search results using arbitrary JSON-serializable key-value metadata.
• ACID Persistence: Crash-safe on-disk embedded database files (.rvf).
• Idiomatic Elixir: First-class structs (RuvectorElixir.VectorEntry), comprehensive typespecs, and functional APIs.

Installation

Add ruvector_elixir to your list of dependencies in mix.exs:

def deps do
  [
    {:ruvector_elixir, "~> 0.1.0"}
  ]
end

System Requirements

A working Rust toolchain (Rust 1.75+ or later) is required to compile the native extension:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Quickstart

1. Opening a Database

Create or open a database with a specified vector dimension (e.g. 128 dimensions):

# Default Cosine metric with exact flat indexing
{:ok, db} = RuvectorElixir.open("priv/data/embeddings.rvf", 128)

# With HNSW index and Euclidean distance
{:ok, db} = RuvectorElixir.open("priv/data/embeddings.rvf", 128, %{
  metric: :euclidean,
  hnsw: %{
    m: 16,
    ef_construction: 100,
    ef_search: 50,
    max_elements: 100_000
  }
})

2. Inserting Vectors

Vectors can be inserted individually or in bulk using RuvectorElixir.VectorEntry structs, maps, or raw lists of floats:

alias RuvectorElixir.VectorEntry

# Insert with explicit ID and metadata using VectorEntry
entry = VectorEntry.new([0.1, 0.2, 0.3, ...], id: "doc_1", metadata: %{
  "title" => "Elixir Guide",
  "category" => "programming",
  "views" => 1200
})
{:ok, "doc_1"} = RuvectorElixir.insert(db, entry)

# Insert with auto-generated UUID
{:ok, id} = RuvectorElixir.insert(db, [0.1, 0.2, 0.3, ...])

# Bulk insert
entries = [
  %{id: "doc_2", vector: [...], metadata: %{"category" => "news"}},
  %{id: "doc_3", vector: [...], metadata: %{"category" => "programming"}}
]
{:ok, ["doc_2", "doc_3"]} = RuvectorElixir.insert_batch(db, entries)

3. Searching for Nearest Neighbors

Perform top-k similarity search using a query vector:

query = [0.1, 0.2, 0.3, ...]

# Fast ID-only search
ids = RuvectorElixir.search(db, query, 5)
# => ["doc_1", "doc_3", "doc_2"]

# Search with metadata filtering
filtered_ids = RuvectorElixir.search(db, query, %{"category" => "programming"}, 5)
# => ["doc_1", "doc_3"]

# Detailed search (returns ID, distance score, vector, and metadata)
{:ok, results} = RuvectorElixir.search_detailed(db, query, %{"category" => "programming"}, 5)
Enum.each(results, fn r ->
  IO.puts("ID: #{r.id} (Score: #{r.score}) - Title: #{r.metadata["title"]}")
end)

4. Fetching, Deleting & Inspecting

# Retrieve vector and metadata by ID
{:ok, entry} = RuvectorElixir.get(db, "doc_1")
IO.inspect(entry.vector)
IO.inspect(entry.metadata)

# Check count and keys
{:ok, count} = RuvectorElixir.len(db)
{:ok, all_ids} = RuvectorElixir.keys(db)
{:ok, false} = RuvectorElixir.empty?(db)

# Delete an entry
{:ok, true} = RuvectorElixir.delete(db, "doc_1")

# Database info
info = RuvectorElixir.info(db)
# => %{"dimensions" => 128, "distance_metric" => "euclidean", "node_count" => 2}

5. Distance Calculations

Direct vector-to-vector distance calculations without opening a database:

v1 = [1.0, 0.0, 0.0]
v2 = [0.0, 1.0, 0.0]

{:ok, dist} = RuvectorElixir.distance(v1, v2, :cosine)
# => {:ok, 1.0}

{:ok, dist} = RuvectorElixir.calculate_distance(v1, v2, :euclidean)
# => {:ok, 1.4142135}

Configuration Options

Database Options

HNSW Configuration

Architecture

RuvectorElixir communicates with ruvector-core via a native C-ABI bridge built with Rustler:

+-------------------------------------------------------------+
|                      Elixir Application                     |
|                   RuvectorElixir (BEAM)                     |
+------------------------------+------------------------------+
                               |
                   Rustler NIF Interface
                               |
               +---------------+---------------+
               |                               |
       [DirtyIo Threads]               [DirtyCpu Threads]
     - open_db / persistence         - k-NN search
     - insert / insert_batch         - distance calculations
     - get / delete / all_ids        - HNSW graph traversal
               |                               |
               +---------------+---------------+
                               |
                    ruvector-core (Rust)
                    - VectorDB & Sled Storage
                    - HNSW & Flat Indexing
                    - SIMD Distance Metrics

• Thread Safety: The Rust database engine features concurrent storage and internal index locking, managed directly inside a Rustler ResourceArc. Multiple Elixir processes (e.g. concurrent Tasks or GenServers) can concurrently read, insert, and search the database with zero locking overhead.
• Zero BEAM Blocking: All CPU-intensive searches run on Erlang DirtyCpu schedulers, while all disk I/O operations run on DirtyIo schedulers, keeping the normal BEAM schedulers free for Web / LiveView requests.

• Idiomatic APIs & Bang Variants: Supports both tuple results {:ok, res} | {:error, reason} and bang variants (open!, insert!, search!, get!, delete!, len!) that raise on failure. Options can be supplied as maps or keyword lists.

Performance & Benchmarks

Because ruvector leverages SIMD instructions (AVX-512, AVX2, NEON) in Rust, vector operations achieve sub-millisecond query latencies:

• Flat Search: Exact search over 10,000 128-dimensional vectors in ~1.5ms.
• HNSW Search: Approximate search over 100,000+ vectors in < 0.2ms with >99% recall.
• Batch Insertion: Up to 100,000 vectors/sec with concurrent pipeline insertion.

Repository & Development

This repository tracks the official upstream ruvector repository as a Git submodule located at reference/ruvector.

Cloning with Submodule

git clone --recurse-submodules https://github.com/ruvnet/ruvector.git
# Or if already cloned:
git submodule update --init --recursive

Running Tests

Execute the complete test suite:

mix test

Generate ExDoc documentation:

mix docs

Upstream Project

• Core Repository: https://github.com/ruvnet/ruvector
• Crates: ruvector-core, ruvector-graph

License

This project is licensed under the MIT License - see the LICENSE file for details.