View Source Vettore (Vettore v0.2.2)
The Vettore library is designed for fast, in-memory operations on vector (embedding) data.
All vectors (embeddings) are stored in a Rust data structure (a HashMap
), accessed via a shared resource
(using Rustler’s ResourceArc
with a Mutex
). Core operations include:
- Creating a collection :
A named set of embeddings with a fixed dimension and a chosen similarity metric (:cosine, :euclidean, :dot,
:hnsw, :binary).
- Inserting an embedding :
Add a new embedding (with ID, vector, and optional metadata) to a specific collection.
- Retrieving embeddings :
Fetch all embeddings from a collection or look up a single embedding by its unique ID.
- Similarity search :
Given a query vector, calculate a “score” for every embedding in the collection and return the top‑k results
(e.g. the smallest distances or largest similarities).
# Usage Example
db = Vettore.new()
:ok = Vettore.create_collection(db, "my_collection", 3, :euclidean)
# Insert an embedding via struct:
embedding = %Vettore.Embedding{value: "my_id or text", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}
:ok = Vettore.insert(db, "my_collection", embedding)
# Retrieve it back:
{:ok, returned_emb} = Vettore.get_by_value(db, "my_collection", "my_id")
IO.inspect(returned_emb.vector, label: "Retrieved vector")
# Perform a similarity search:
{:ok, top_results} = Vettore.similarity_search(db, "my_collection", [1.5, 1.5, 1.5], 2)
IO.inspect(top_results, label: "Top K search results")
Link to this section Summary
Functions
Batch‑insert a list of embeddings. Reject elements that are not %Vettore.Embedding{}
.
Create a collection.
Delete a single embedding.
Delete a collection.
Return all embeddings in raw form ({value, vector, metadata}
tuples).
Fetch a single embedding by value (ID) and return it as %Vettore.Embedding{}
.
Fetch a single embedding by vector and return it as %Vettore.Embedding{}
.
Insert one %Vettore.Embedding{}
into the collection.
Returns {:ok, value}
on success or {:error, reason}
.
Allocate an empty in‑memory DB (owned by Rust). Keep the returned reference around – every other call expects it.
Re‑rank an existing result list with Maximal Marginal Relevance.
Similarity / nearest‑neighbour search.
Link to this section Functions
@spec batch(reference(), String.t(), [Vettore.Embedding.t()]) :: {:ok, [String.t()]} | {:error, String.t()}
Batch‑insert a list of embeddings. Reject elements that are not %Vettore.Embedding{}
.
Examples
iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.batch("my_collection", [%Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}])
{:ok, ["my_id"]}
@spec create_collection( reference(), String.t(), pos_integer(), atom(), keyword() ) :: {:ok, String.t()} | {:error, String.t()}
Create a collection.
distance
must be one of the atoms::euclidean
,:cosine
,:dot
,:hnsw
, or:binary
.
Examples
iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean)
{:ok, "my_collection"}
Delete a single embedding.
Examples
iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.delete("my_collection", "my_id")
{:ok, "my_id"}
Delete a collection.
Examples
iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.delete_collection("my_collection")
{:ok, "my_collection"}
@spec get_all(reference(), String.t()) :: {:ok, [{String.t(), [number()], map() | nil}]} | {:error, String.t()}
Return all embeddings in raw form ({value, vector, metadata}
tuples).
Examples
iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.get_all("my_collection")
{:ok, [{"my_id", [1.0, 2.0, 3.0], %{"note" => "hello"}}]}
@spec get_by_value(reference(), String.t(), String.t()) :: {:ok, Vettore.Embedding.t()} | {:error, String.t()}
Fetch a single embedding by value (ID) and return it as %Vettore.Embedding{}
.
Examples
iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.get_by_value("my_collection", "my_id")
{:ok, %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}}
@spec get_by_vector(reference(), String.t(), [number()]) :: {:ok, Vettore.Embedding.t()} | {:error, String.t()}
Fetch a single embedding by vector and return it as %Vettore.Embedding{}
.
Examples
iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.get_by_vector("my_collection", [1.0, 2.0, 3.0])
{:ok, %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}}
@spec insert(reference(), String.t(), Vettore.Embedding.t()) :: {:ok, String.t()} | {:error, String.t()}
Insert one %Vettore.Embedding{}
into the collection.
Returns {:ok, value}
on success or {:error, reason}
.
Examples
iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}})
{:ok, "my_id"}
@spec new() :: reference()
Allocate an empty in‑memory DB (owned by Rust). Keep the returned reference around – every other call expects it.
@spec rerank(reference(), String.t(), [{String.t(), number()}], keyword()) :: {:ok, [{String.t(), number()}]} | {:error, String.t()}
Re‑rank an existing result list with Maximal Marginal Relevance.
Options:
:limit
– desired output length (default 10):alpha
– relevance‑diversity balance 0.0..1.0 (default 0.5)
# Examples
iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id2", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id3", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.rerank("my_collection", [{"my_id", 0.0}, {"my_id2", 0.0}, {"my_id3", 0.0}], limit: 1)
{:ok, [{"my_id", 0.0}]}
@spec similarity_search(reference(), String.t(), [number()], keyword()) :: {:ok, [{String.t(), float()}]} | {:error, String.t()}
Similarity / nearest‑neighbour search.
Options:
:limit
– number of results (default 10):filter
– metadata map; only embeddings whose metadata contains all key‑value pairs are considered.
Examples
iex> Vettore.new() |> Vettore.create_collection("my_collection", 3, :euclidean) |> Vettore.insert("my_collection", %Vettore.Embedding{value: "my_id", vector: [1.0, 2.0, 3.0], metadata: %{"note" => "hello"}}) |> Vettore.similarity_search("my_collection", [1.0, 2.0, 3.0], limit: 1)
{:ok, [{"my_id", 0.0}]}