RuvectorElixir (ruvector_elixir v0.1.0)

Copy Markdown View Source

High-performance Elixir NIF bindings to ruvector-core, an embedded vector database written in Rust.

Provides sub-millisecond approximate nearest neighbor (ANN) search via HNSW or exact flat indexing, supporting multiple distance metrics (cosine, euclidean, dot product, manhattan), metadata filtering, and ACID-compliant on-disk persistence as well as pure in-memory databases (memory://).

Architecture

RuvectorElixir uses Rustler to interface directly with the ruvector-core Rust engine via dirty schedulers (DirtyIo and DirtyCpu) to avoid blocking the Erlang VM's normal scheduler threads.

Features

  • Vector Operations: Insertion (single & batch), retrieval by ID, deletion, key enumeration.
  • Distance Metrics: Cosine similarity, Euclidean (L2), Dot product, Manhattan (L1).
  • Search: Fast k-nearest neighbor search with optional metadata JSON filtering and ef_search parameter.
  • HNSW Indexing: Configurable m, ef_construction, and ef_search parameters for graph-based ANN indexing.
  • In-Memory & Persistent Storage: Sled/disk backed .rvf storage or pure in-memory memory://<name> databases.
  • Quantization: Support for scalar and binary quantization configurations.

Summary

Functions

Returns a list of all vector IDs currently stored in the database.

Calculates the distance between two vectors a and b using the specified metric.

Convenience alias for db_len/1.

Convenience alias for len!/1.

Returns the total number of vectors in the database.

Deletes an entry by its ID (string, atom, or integer).

Deletes an entry by its ID, raising RuntimeError on failure.

Convenience helper for calculate_distance/3, defaulting to :cosine.

Calculates distance between two vectors, raising ArgumentError on failure.

Convenience alias for is_empty/1.

Convenience alias for get_entry/2.

Convenience alias for get_entry!/2.

Retrieves a vector entry by its ID (string, atom, or integer).

Retrieves a vector entry, raising RuntimeError if not found or if an error occurs.

Retrieves a vector entry and returns it as a RuvectorElixir.VectorEntry struct.

Retrieves a vector entry as a VectorEntry struct, raising RuntimeError if not found or on error.

Gets database metadata info including :dimensions, :node_count, :distance_metric, :storage_path, and :hnsw.

Inserts a vector entry into the database.

Inserts a vector entry into the database, raising ArgumentError on failure.

Inserts a batch of vector entries into the database.

Inserts a batch of vector entries into the database, raising ArgumentError on failure.

Checks if the database is empty.

Convenience alias for all_ids/1.

Returns all vector IDs in the database, raising on error.

Convenience alias for db_len/1.

Returns the total number of vectors in the database, raising on error.

Opens or creates a ruvector database at path with the specified vector dimensions and optional configuration options.

Opens a database, raising RuntimeError on failure.

Opens a ruvector database at path with dimensions.

Opens a ruvector database with configuration options.

Native NIF: Opens a ruvector database with optional settings map.

Searches the database for the top k closest vectors to query_vector.

Searches the database for the top k closest vectors matching the optional filter.

Searches the database for top k closest vectors, raising ArgumentError on failure.

Searches the database for top k closest vectors with filter, raising ArgumentError on failure.

Searches the database for top k closest vectors, returning detailed search result maps.

Searches the database with a metadata filter, returning detailed search results.

Native NIF: Searches the database with a filter and optional HNSW ef_search override.

Searches the database returning detailed search results, raising ArgumentError on failure.

Types

db_options()

@type db_options() :: %{
  optional(:metric) => metric(),
  optional(:distance_metric) => metric(),
  optional(:hnsw) => boolean() | hnsw_options() | keyword(),
  optional(:quantization) => quantization_options() | keyword()
}

db_ref()

@type db_ref() :: reference()

entry_input()

@type entry_input() ::
  RuvectorElixir.VectorEntry.t()
  | %{
      optional(:id) => id_input(),
      optional(:vector) => vector(),
      optional(:metadata) => map() | nil
    }
  | vector()

entry_map()

@type entry_map() :: %{id: String.t(), vector: [float()], metadata: map() | nil}

hnsw_options()

@type hnsw_options() :: %{
  optional(:m) => pos_integer(),
  optional(:ef_construction) => pos_integer(),
  optional(:ef_search) => pos_integer(),
  optional(:max_elements) => pos_integer()
}

id_input()

@type id_input() :: String.t() | atom() | integer()

metric()

@type metric() :: :cosine | :euclidean | :dot_product | :dot | :manhattan | String.t()

quantization_options()

@type quantization_options() ::
  :scalar
  | :binary
  | :none
  | String.t()
  | %{
      optional(:type) => :scalar | :binary | :product | String.t(),
      optional(:subspaces) => pos_integer(),
      optional(:k) => pos_integer()
    }

search_result()

@type search_result() :: %{
  id: String.t(),
  score: float(),
  vector: [float()] | nil,
  metadata: map() | nil
}

vector()

@type vector() :: [number()]

Functions

all_ids(db_ref)

@spec all_ids(db_ref()) :: {:ok, [String.t()]} | {:error, String.t()}

Returns a list of all vector IDs currently stored in the database.

calculate_distance(a, b, metric)

@spec calculate_distance(vector(), vector(), metric()) ::
  {:ok, float()} | {:error, String.t()}

Calculates the distance between two vectors a and b using the specified metric.

Supported metrics: :cosine, :euclidean, :dot_product, :manhattan.

Returns {:ok, distance} on success, or {:error, reason} on failure (e.g. dimension mismatch).

count(db_ref)

@spec count(db_ref()) :: {:ok, non_neg_integer()} | {:error, String.t()}

Convenience alias for db_len/1.

count!(db_ref)

@spec count!(db_ref()) :: non_neg_integer()

Convenience alias for len!/1.

db_len(db_ref)

@spec db_len(db_ref()) :: {:ok, non_neg_integer()} | {:error, String.t()}

Returns the total number of vectors in the database.

delete(db_ref, id)

@spec delete(db_ref(), id_input()) :: {:ok, boolean()} | {:error, String.t()}

Deletes an entry by its ID (string, atom, or integer).

Returns {:ok, true} if deleted, {:ok, false} if not found, or {:error, reason} on failure.

delete!(db_ref, id)

@spec delete!(db_ref(), id_input()) :: boolean()

Deletes an entry by its ID, raising RuntimeError on failure.

distance(a, b, metric \\ :cosine)

@spec distance(vector(), vector(), metric()) :: {:ok, float()} | {:error, String.t()}

Convenience helper for calculate_distance/3, defaulting to :cosine.

distance!(a, b, metric \\ :cosine)

@spec distance!(vector(), vector(), metric()) :: float()

Calculates distance between two vectors, raising ArgumentError on failure.

empty?(db_ref)

@spec empty?(db_ref()) :: {:ok, boolean()} | {:error, String.t()}

Convenience alias for is_empty/1.

get(db_ref, id)

@spec get(db_ref(), id_input()) ::
  {:ok, entry_map()} | {:error, :not_found} | {:error, String.t()}

Convenience alias for get_entry/2.

get!(db_ref, id)

@spec get!(db_ref(), id_input()) :: entry_map()

Convenience alias for get_entry!/2.

get_entry(db_ref, id)

@spec get_entry(db_ref(), id_input()) ::
  {:ok, entry_map()} | {:error, :not_found} | {:error, String.t()}

Retrieves a vector entry by its ID (string, atom, or integer).

Returns {:ok, %{id: id, vector: vector, metadata: metadata}} if found, {:error, :not_found} if no entry with that ID exists, or {:error, reason} on failure.

get_entry!(db_ref, id)

@spec get_entry!(db_ref(), id_input()) :: entry_map()

Retrieves a vector entry, raising RuntimeError if not found or if an error occurs.

get_struct(db_ref, id)

@spec get_struct(db_ref(), id_input()) ::
  {:ok, RuvectorElixir.VectorEntry.t()}
  | {:error, :not_found}
  | {:error, String.t()}

Retrieves a vector entry and returns it as a RuvectorElixir.VectorEntry struct.

get_struct!(db_ref, id)

@spec get_struct!(db_ref(), id_input()) :: RuvectorElixir.VectorEntry.t()

Retrieves a vector entry as a VectorEntry struct, raising RuntimeError if not found or on error.

info(db_ref)

@spec info(db_ref()) :: map()

Gets database metadata info including :dimensions, :node_count, :distance_metric, :storage_path, and :hnsw.

insert(db_ref, entry)

@spec insert(db_ref(), entry_input()) :: {:ok, String.t()} | {:error, String.t()}

Inserts a vector entry into the database.

The entry can be:

  • A RuvectorElixir.VectorEntry struct.
  • A map with :vector (and optional :id, :metadata).
  • A raw list of numbers [1.0, 2.0, ...] (in which case a UUID is automatically assigned).

Returns {:ok, id} on success, or {:error, reason} on failure.

Examples

{:ok, id} = RuvectorElixir.insert(db, [0.1, 0.2, 0.3])
{:ok, "doc_1"} = RuvectorElixir.insert(db, %{id: "doc_1", vector: [0.1, 0.2, 0.3], metadata: %{"author" => "Alice"}})

insert!(db_ref, entry)

@spec insert!(db_ref(), entry_input()) :: String.t()

Inserts a vector entry into the database, raising ArgumentError on failure.

insert_batch(db_ref, entries)

@spec insert_batch(db_ref(), [entry_input()]) ::
  {:ok, [String.t()]} | {:error, String.t()}

Inserts a batch of vector entries into the database.

Returns {:ok, [id]} on success, or {:error, reason} on failure.

Examples

entries = [
  %{id: "v1", vector: [0.1, 0.2, 0.3]},
  %{id: "v2", vector: [0.4, 0.5, 0.6]}
]
{:ok, ["v1", "v2"]} = RuvectorElixir.insert_batch(db, entries)

insert_batch!(db_ref, entries)

@spec insert_batch!(db_ref(), [entry_input()]) :: [String.t()]

Inserts a batch of vector entries into the database, raising ArgumentError on failure.

is_empty(db_ref)

@spec is_empty(db_ref()) :: {:ok, boolean()} | {:error, String.t()}

Checks if the database is empty.

keys(db_ref)

@spec keys(db_ref()) :: {:ok, [String.t()]} | {:error, String.t()}

Convenience alias for all_ids/1.

keys!(db_ref)

@spec keys!(db_ref()) :: [String.t()]

Returns all vector IDs in the database, raising on error.

len(db_ref)

@spec len(db_ref()) :: {:ok, non_neg_integer()} | {:error, String.t()}

Convenience alias for db_len/1.

len!(db_ref)

@spec len!(db_ref()) :: non_neg_integer()

Returns the total number of vectors in the database, raising on error.

open(path, dimensions, options \\ %{})

@spec open(Path.t(), pos_integer(), db_options() | keyword()) ::
  {:ok, db_ref()} | {:error, String.t()}

Opens or creates a ruvector database at path with the specified vector dimensions and optional configuration options.

Options can be supplied as either a map or a keyword list.

Options

  • :metric / :distance_metric - Distance metric to use (:cosine, :euclidean, :dot_product, :manhattan). Default: :cosine.
  • :hnsw - Either a boolean true for default HNSW index parameters, or a map/keyword list with:
    • :m - Number of bi-directional links created per node (default: 16).
    • :ef_construction - Size of dynamic candidate list during construction (default: 100).
    • :ef_search - Size of dynamic candidate list during search (default: 50).
    • :max_elements - Initial capacity of the graph index (default: 1_000_000).
  • :quantization - Optional quantization configuration (:scalar, :binary, :none, or product map).

In-Memory Databases

Use a path prefixed with memory:// (e.g. memory://my_session) for a non-persisted in-memory database.

Examples

{:ok, db} = RuvectorElixir.open("my_db.rvf", 128)
{:ok, db} = RuvectorElixir.open("my_db.rvf", 128, metric: :euclidean, hnsw: true)
{:ok, mem_db} = RuvectorElixir.open("memory://transient", 64)

open!(path, dimensions, options \\ %{})

@spec open!(Path.t(), pos_integer(), db_options() | keyword()) :: db_ref()

Opens a database, raising RuntimeError on failure.

open_db(path, dimensions)

@spec open_db(Path.t(), pos_integer()) :: {:ok, db_ref()} | {:error, String.t()}

Opens a ruvector database at path with dimensions.

Equivalent to open(path, dimensions, %{}).

open_db(path, dimensions, options)

@spec open_db(Path.t(), pos_integer(), db_options() | keyword()) ::
  {:ok, db_ref()} | {:error, String.t()}

Opens a ruvector database with configuration options.

open_db_with_options(path, dimensions, options)

@spec open_db_with_options(Path.t(), pos_integer(), db_options() | nil) ::
  {:ok, db_ref()} | {:error, String.t()}

Native NIF: Opens a ruvector database with optional settings map.

search(db_ref, query_vector, k)

@spec search(db_ref(), vector(), pos_integer()) :: [String.t()] | {:error, String.t()}

Searches the database for the top k closest vectors to query_vector.

Returns a list of matching vector ID strings, or {:error, reason} on failure.

search(db_ref, query_vector, filter, k)

@spec search(db_ref(), vector(), map() | keyword() | nil, pos_integer()) ::
  [String.t()] | {:error, String.t()}

Searches the database for the top k closest vectors matching the optional filter.

Filter can be a map or a keyword list.

Returns a list of matching vector ID strings, or {:error, reason} on failure.

search!(db_ref, query_vector, k)

@spec search!(db_ref(), vector(), pos_integer()) :: [String.t()]

Searches the database for top k closest vectors, raising ArgumentError on failure.

search!(db_ref, query_vector, filter, k)

@spec search!(db_ref(), vector(), map() | keyword() | nil, pos_integer()) :: [
  String.t()
]

Searches the database for top k closest vectors with filter, raising ArgumentError on failure.

search_detailed(db_ref, query_vector, k)

@spec search_detailed(db_ref(), vector(), pos_integer()) ::
  {:ok, [search_result()]} | {:error, String.t()}

Searches the database for top k closest vectors, returning detailed search result maps.

Each result map contains:

  • :id - Vector ID string
  • :score - Similarity / distance score (lower is closer for distance metrics)
  • :vector - Vector embedding float list (or nil)
  • :metadata - Metadata map (or nil)

search_detailed(db_ref, query_vector, filter, k)

@spec search_detailed(db_ref(), vector(), map() | keyword() | nil, pos_integer()) ::
  {:ok, [search_result()]} | {:error, String.t()}

Searches the database with a metadata filter, returning detailed search results.

search_detailed(db_ref, query_vector, filter, k, ef_search)

@spec search_detailed(
  db_ref(),
  vector(),
  map() | keyword() | nil,
  pos_integer(),
  pos_integer() | nil
) ::
  {:ok, [search_result()]} | {:error, String.t()}

Native NIF: Searches the database with a filter and optional HNSW ef_search override.

search_detailed!(db_ref, query_vector, filter \\ nil, k, ef_search \\ nil)

@spec search_detailed!(
  db_ref(),
  vector(),
  map() | keyword() | nil,
  pos_integer(),
  pos_integer() | nil
) ::
  [search_result()]

Searches the database returning detailed search results, raising ArgumentError on failure.