RuvectorElixir (ruvector_elixir v0.2.0)

Copy Markdown View Source

High-performance Elixir NIF bindings to ruvector-core, an embedded vector database written in Rust.

Provides sub-millisecond approximate nearest neighbor (ANN) search via HNSW or exact flat indexing, supporting multiple distance metrics (cosine, euclidean, dot product, manhattan), metadata filtering, on-disk persistence via redb, and isolated in-memory databases (memory://).

Architecture

RuvectorElixir uses Rustler to interface directly with ruvector-core on Erlang dirty schedulers (DirtyIo and DirtyCpu) so BEAM normal schedulers are never blocked.

Features

  • Vector Operations: Insertion (single & batch), retrieval by ID, deletion, key enumeration.
  • Distance Metrics: Cosine similarity, Euclidean (L2), Dot product, Manhattan (L1).
  • Search: Fast k-nearest neighbor search with optional metadata filtering and ef_search tuning.
  • HNSW Indexing: Configurable m, ef_construction, and ef_search parameters for graph-based ANN indexing.
  • In-Memory & Persistent Storage: Disk-backed .rvf storage or pure in-memory memory://<name> databases.
  • Explicit Lifecycle: Deterministic close/1 to release database handles immediately.

Summary

Functions

Closes a database handle, releasing its underlying Rust database resources immediately.

Closes a database handle, raising RuvectorElixir.Error on failure.

Convenience alias for len/1.

Convenience alias for len!/1.

Deletes an entry by its ID (string, non-boolean atom, or integer).

Deletes an entry by its ID, raising RuvectorElixir.Error on failure.

Calculates the distance between two vectors a and b using the specified metric.

Calculates distance between two vectors, raising RuvectorElixir.Error on operational failure (e.g. dimension mismatch) or ArgumentError on invalid inputs.

Checks if the database is empty.

Retrieves a vector entry by its ID (string, non-boolean atom, or integer).

Retrieves a vector entry, raising RuvectorElixir.Error if not found or if an error occurs.

Retrieves a vector entry and returns it as a RuvectorElixir.VectorEntry struct.

Retrieves a vector entry as a VectorEntry struct, raising RuvectorElixir.Error on failure.

Gets database metadata info including :dimensions, :node_count, :distance_metric, :storage_path, :hnsw, and :quantization.

Gets database metadata info, raising RuvectorElixir.Error on failure.

Inserts a vector entry into the database.

Inserts a vector entry into the database, raising RuvectorElixir.Error on failure.

Inserts a batch of vector entries into the database.

Inserts a batch of vector entries into the database, raising RuvectorElixir.Error on failure.

Returns a list of all vector IDs currently stored in the database.

Returns all vector IDs in the database, raising RuvectorElixir.Error on error.

Returns the total number of vectors in the database.

Returns the total number of vectors in the database, raising RuvectorElixir.Error on error.

Opens or creates a ruvector database at path with the specified vector dimensions and optional configuration options.

Opens a database, raising RuvectorElixir.Error on operational failure or ArgumentError on invalid arguments.

Searches the database for the top k closest vectors to query_vector.

Searches the database for top k closest vectors, raising RuvectorElixir.Error on operational failure or ArgumentError on invalid arguments.

Searches the database for top k closest vectors, returning detailed search result maps.

Searches the database returning detailed search results, raising RuvectorElixir.Error on operational failure or ArgumentError on invalid arguments.

Opens a database, passes the handle to fun, and ensures close/1 runs after fun completes or raises.

Opens a database with open!/3, passes the handle to fun, ensures close/1 runs, and returns the result of fun directly.

Types

db_info()

@type db_info() :: %{
  node_count: non_neg_integer(),
  dimensions: pos_integer(),
  distance_metric: :cosine | :euclidean | :dot_product | :manhattan,
  storage_path: String.t(),
  hnsw: hnsw_options() | nil,
  quantization:
    :none
    | :scalar
    | :binary
    | %{type: :product, subspaces: pos_integer(), k: pos_integer()}
    | nil
}

db_options()

@type db_options() ::
  %{
    optional(:metric) => metric(),
    optional(:distance_metric) => metric(),
    optional(:hnsw) =>
      boolean() | hnsw_options() | %{optional(String.t()) => term()} | keyword(),
    optional(:quantization) => quantization_options()
  }
  | %{optional(String.t()) => term()}

db_ref()

@opaque db_ref()

entry_input()

@type entry_input() ::
  RuvectorElixir.VectorEntry.t()
  | %{
      :vector => vector(),
      optional(:id) => id_input() | nil,
      optional(:metadata) => map() | nil
    }
  | %{required(String.t()) => vector() | id_input() | map() | nil}
  | vector()

entry_map()

@type entry_map() :: %{id: String.t(), vector: [float()], metadata: map() | nil}

hnsw_options()

@type hnsw_options() :: %{
  optional(:m) => pos_integer(),
  optional(:ef_construction) => pos_integer(),
  optional(:ef_search) => pos_integer(),
  optional(:max_elements) => pos_integer()
}

id_input()

@type id_input() :: String.t() | atom() | integer()

metric()

@type metric() :: :cosine | :euclidean | :dot_product | :dot | :manhattan | String.t()

quantization_options()

@type quantization_options() ::
  :scalar
  | :binary
  | :none
  | String.t()
  | keyword()
  | %{
      optional(:type) => :scalar | :binary | :none | :product | String.t(),
      optional(:subspaces) => pos_integer(),
      optional(:k) => pos_integer()
    }
  | %{optional(String.t()) => term()}

search_detailed_option()

@type search_detailed_option() :: search_option()

search_filter()

@type search_filter() ::
  %{optional(String.t() | atom()) => term()}
  | [{String.t() | atom(), term()}]
  | nil

search_option()

@type search_option() ::
  {:filter, search_filter()} | {:ef_search, pos_integer() | nil}

search_options()

@type search_options() ::
  [search_option()]
  | %{
      optional(:filter) => search_filter(),
      optional(:ef_search) => pos_integer() | nil
    }
  | %{optional(String.t()) => search_filter() | pos_integer() | nil}

search_result()

@type search_result() :: %{
  id: String.t(),
  score: float(),
  vector: [float()] | nil,
  metadata: map() | nil
}

vector()

@type vector() :: [number()]

Functions

close(db_ref)

@spec close(db_ref()) :: :ok | {:error, String.t()}

Closes a database handle, releasing its underlying Rust database resources immediately.

Idempotent: calling close/1 on an already-closed handle returns :ok. Subsequent operations on a closed handle return {:error, :closed}.

close!(db_ref)

@spec close!(db_ref()) :: :ok

Closes a database handle, raising RuvectorElixir.Error on failure.

count(db_ref)

@spec count(db_ref()) :: {:ok, non_neg_integer()} | {:error, :closed | String.t()}

Convenience alias for len/1.

count!(db_ref)

@spec count!(db_ref()) :: non_neg_integer()

Convenience alias for len!/1.

delete(db_ref, id)

@spec delete(db_ref(), id_input()) ::
  {:ok, boolean()} | {:error, :closed | String.t()}

Deletes an entry by its ID (string, non-boolean atom, or integer).

Returns {:ok, true} if deleted, {:ok, false} if not found, or {:error, reason} on failure.

delete!(db_ref, id)

@spec delete!(db_ref(), id_input()) :: boolean()

Deletes an entry by its ID, raising RuvectorElixir.Error on failure.

distance(a, b, metric \\ :cosine)

@spec distance(vector(), vector(), metric()) :: {:ok, float()} | {:error, String.t()}

Calculates the distance between two vectors a and b using the specified metric.

Supported metrics: :cosine, :euclidean, :dot_product, :dot, :manhattan. Defaults to :cosine.

Returns {:ok, distance} on success, or {:error, reason} on failure (e.g. dimension mismatch).

distance!(a, b, metric \\ :cosine)

@spec distance!(vector(), vector(), metric()) :: float()

Calculates distance between two vectors, raising RuvectorElixir.Error on operational failure (e.g. dimension mismatch) or ArgumentError on invalid inputs.

empty?(db_ref)

@spec empty?(db_ref()) :: boolean()

Checks if the database is empty.

Returns true or false, raising RuvectorElixir.Error on failure (such as :closed). Callers needing a non-raising tuple can check len/1 (matching {:ok, 0}).

get(db_ref, id)

@spec get(db_ref(), id_input()) ::
  {:ok, entry_map()} | {:error, :not_found | :closed | String.t()}

Retrieves a vector entry by its ID (string, non-boolean atom, or integer).

Returns {:ok, %{id: id, vector: vector, metadata: metadata}} if found, {:error, :not_found} if no entry with that ID exists, {:error, :closed} if closed, or {:error, reason} on failure.

get!(db_ref, id)

@spec get!(db_ref(), id_input()) :: entry_map()

Retrieves a vector entry, raising RuvectorElixir.Error if not found or if an error occurs.

get_struct(db_ref, id)

@spec get_struct(db_ref(), id_input()) ::
  {:ok, RuvectorElixir.VectorEntry.t()}
  | {:error, :not_found | :closed | String.t()}

Retrieves a vector entry and returns it as a RuvectorElixir.VectorEntry struct.

get_struct!(db_ref, id)

@spec get_struct!(db_ref(), id_input()) :: RuvectorElixir.VectorEntry.t()

Retrieves a vector entry as a VectorEntry struct, raising RuvectorElixir.Error on failure.

info(db_ref)

@spec info(db_ref()) :: {:ok, db_info()} | {:error, :closed | String.t()}

Gets database metadata info including :dimensions, :node_count, :distance_metric, :storage_path, :hnsw, and :quantization.

info!(db_ref)

@spec info!(db_ref()) :: db_info()

Gets database metadata info, raising RuvectorElixir.Error on failure.

insert(db_ref, entry)

@spec insert(db_ref(), entry_input()) ::
  {:ok, String.t()} | {:error, :closed | String.t()}

Inserts a vector entry into the database.

The entry can be:

  • A RuvectorElixir.VectorEntry struct.
  • A map with :vector (and optional :id, :metadata).
  • A raw list of numbers [1.0, 2.0, ...] (in which case a UUID is automatically assigned).

Returns {:ok, id} on success, or {:error, reason} on failure.

insert!(db_ref, entry)

@spec insert!(db_ref(), entry_input()) :: String.t()

Inserts a vector entry into the database, raising RuvectorElixir.Error on failure.

insert_batch(db_ref, entries)

@spec insert_batch(db_ref(), [entry_input()]) ::
  {:ok, [String.t()]} | {:error, :closed | String.t()}

Inserts a batch of vector entries into the database.

Returns {:ok, [id]} on success, or {:error, reason} on failure.

insert_batch!(db_ref, entries)

@spec insert_batch!(db_ref(), [entry_input()]) :: [String.t()]

Inserts a batch of vector entries into the database, raising RuvectorElixir.Error on failure.

keys(db_ref)

@spec keys(db_ref()) :: {:ok, [String.t()]} | {:error, :closed | String.t()}

Returns a list of all vector IDs currently stored in the database.

keys!(db_ref)

@spec keys!(db_ref()) :: [String.t()]

Returns all vector IDs in the database, raising RuvectorElixir.Error on error.

len(db_ref)

@spec len(db_ref()) :: {:ok, non_neg_integer()} | {:error, :closed | String.t()}

Returns the total number of vectors in the database.

len!(db_ref)

@spec len!(db_ref()) :: non_neg_integer()

Returns the total number of vectors in the database, raising RuvectorElixir.Error on error.

open(path, dimensions, options \\ %{})

@spec open(String.t(), pos_integer(), db_options() | keyword()) ::
  {:ok, db_ref()} | {:error, String.t()}

Opens or creates a ruvector database at path with the specified vector dimensions and optional configuration options.

Options can be supplied as either a plain map or a keyword list. Invalid option keys or values raise ArgumentError. When reopening an existing .rvf database on disk, ruvector-core uses the persisted configuration (dimensions, distance_metric, hnsw, and quantization) already stored in the file.

Options

  • :metric / :distance_metric - Distance metric to use (:cosine, :euclidean, :dot_product, :manhattan). Default: :cosine.
  • :hnsw - Either a boolean (true for default HNSW index parameters, false for flat index), or a map/keyword list with:
    • :m - Number of bi-directional links created per node (2..256, default: 16).
    • :ef_construction - Size of dynamic candidate list during construction (1..1_000_000, default: 100).
    • :ef_search - Size of dynamic candidate list during search (1..1_000_000, default: 100).
    • :max_elements - Initial capacity of the graph index (1..10_000_000, default: 1_000_000).
  • :quantization - Optional quantization configuration (:scalar, :binary, :none, or %{type: :product, subspaces: pos_integer(), k: pos_integer()} where subspaces must evenly divide dimensions). Note: upstream ruvector-core 2.3.0 persists and reports :quantization in config/info, but stores vectors in f32 precision without applying compression yet (ruvnet/RuVector#563).

Vectors are stored internally as 32-bit IEEE 754 floats (f32); non-dyadic 64-bit Elixir floats are rounded to f32 precision on insert and widened back to f64 on read.

In-Memory Databases

Use a path prefixed with memory:// (e.g. "memory://transient") for a pure in-memory database that does not touch the filesystem. Every open("memory://...", ...) call allocates an independent, isolated in-memory database instance regardless of the URI suffix.

Examples

iex> {:ok, mem_db} = RuvectorElixir.open("memory://transient", 64)
iex> {:ok, %{dimensions: 64}} = RuvectorElixir.info(mem_db)
iex> RuvectorElixir.close(mem_db)
:ok

On-disk databases use a filesystem path (close an open handle before reopening the same path):

{:ok, db} = RuvectorElixir.open("my_db.rvf", 128)
:ok = RuvectorElixir.close(db)
{:ok, hnsw_db} = RuvectorElixir.open("hnsw_db.rvf", 128, metric: :euclidean, hnsw: true)
:ok = RuvectorElixir.close(hnsw_db)

open!(path, dimensions, options \\ %{})

@spec open!(String.t(), pos_integer(), db_options() | keyword()) :: db_ref()

Opens a database, raising RuvectorElixir.Error on operational failure or ArgumentError on invalid arguments.

search(db_ref, query_vector, k, opts \\ [])

@spec search(db_ref(), vector(), non_neg_integer(), search_options()) ::
  {:ok, [String.t()]} | {:error, :closed | String.t()}

Searches the database for the top k closest vectors to query_vector.

Options

  • :filter - Optional metadata filter map or keyword list (default: nil).
  • :ef_search - Optional positive integer HNSW ef_search override (default: nil).

Returns {:ok, [id]} on success, or {:error, reason} on failure.

search!(db_ref, query_vector, k, opts \\ [])

@spec search!(db_ref(), vector(), non_neg_integer(), search_options()) :: [String.t()]

Searches the database for top k closest vectors, raising RuvectorElixir.Error on operational failure or ArgumentError on invalid arguments.

search_detailed(db_ref, query_vector, k, opts \\ [])

@spec search_detailed(db_ref(), vector(), non_neg_integer(), search_options()) ::
  {:ok, [search_result()]} | {:error, :closed | String.t()}

Searches the database for top k closest vectors, returning detailed search result maps.

Options

  • :filter - Optional metadata filter map or keyword list (default: nil).
  • :ef_search - Optional positive integer HNSW ef_search override (default: nil).

Each result map contains:

  • :id - Vector ID string
  • :score - Similarity / distance score (lower is closer for distance metrics)
  • :vector - Vector embedding float list (or nil)
  • :metadata - Metadata map (or nil)

search_detailed!(db_ref, query_vector, k, opts \\ [])

@spec search_detailed!(db_ref(), vector(), non_neg_integer(), search_options()) :: [
  search_result()
]

Searches the database returning detailed search results, raising RuvectorElixir.Error on operational failure or ArgumentError on invalid arguments.

with_open(path, dimensions, fun)

@spec with_open(String.t(), pos_integer(), (db_ref() -> result)) ::
  {:ok, result} | {:error, String.t()}
when result: var

Opens a database, passes the handle to fun, and ensures close/1 runs after fun completes or raises.

Returns {:ok, result} on success, or {:error, reason} if open/3 fails.

with_open(path, dimensions, opts, fun)

@spec with_open(String.t(), pos_integer(), db_options() | keyword(), (db_ref() ->
                                                                  result)) ::
  {:ok, result} | {:error, String.t()}
when result: var

with_open!(path, dimensions, fun)

@spec with_open!(String.t(), pos_integer(), (db_ref() -> result)) :: result
when result: var

Opens a database with open!/3, passes the handle to fun, ensures close/1 runs, and returns the result of fun directly.

with_open!(path, dimensions, opts, fun)

@spec with_open!(String.t(), pos_integer(), db_options() | keyword(), (db_ref() ->
                                                                   result)) ::
  result
when result: var