Module kvex

kvex — pure Erlang approximate k-NN vector search on the BEAM.

Description

kvex — pure Erlang approximate k-NN vector search on the BEAM.

Two ETS tables per index: • vec table — {Id, F32Bin, BinVec} per vector (source of truth) • flat cache — {flat, F32FlatBin, BvecFlatBin, IdsTuple} rebuilt on every insert; a single refc-binary per flat, so search never iterates over individual Erlang terms.

Search path (two sied NIF calls on flat binaries, no Erlang list work): 1. sied:hamming_topk_flat/4 — SIMD POPCNT on BvecFlat, O(N)+O(K logK) 2. sied:dot_product_topk_flat/4 — SIMD dot-product on F32Flat candidates

Quick start

   {ok, Ix} = kvex:new(128),
   Vec      = [rand:uniform() || _ <- lists:seq(1, 128)],
   ok       = kvex:add(Ix, 42, Vec),
   {ok, Rs} = kvex:search(Ix, Vec, 5),
   ok       = kvex:delete(Ix).

Data Types

id()

id() = non_neg_integer() | binary()

index()

abstract datatype: index()

opts()

opts() = #{bits => 2 | 3 | 4}

vector()

vector() = [float()] | binary()

Function Index

add/3Inserts a single vector.
add_batch/2Inserts vectors in batch.
cosine_search/3
delete/1
new/1
new/2Creates an empty index for vectors of dimension Dim.
normalize/1
search/3
size/1
version/0

Function Details

add/3

add(X1::index(), Id::id(), Vec0::vector()) -> ok | {error, term()}

Inserts a single vector. Rebuilds the flat cache — O(N) copy.

add_batch/2

add_batch(X1::index(), Pairs::[{id(), vector()}]) -> ok | {error, term()}

Inserts vectors in batch. Builds the flat binary incrementally — O(batch).

cosine_search/3

cosine_search(Ix::index(), Query::vector(), K::pos_integer()) -> {ok, [{id(), Score::float()}]} | {error, term()}

delete/1

delete(X1::index()) -> ok

new/1

new(Dim::pos_integer()) -> {ok, index()} | {error, term()}

new/2

new(Dim::pos_integer(), Opts::opts()) -> {ok, index()} | {error, term()}

Creates an empty index for vectors of dimension Dim.

normalize/1

normalize(Vec::vector()) -> {ok, [float()]} | {error, term()}

search/3

search(X1::index(), Query::vector(), K::pos_integer()) -> {ok, [{id(), Score::float()}]} | {error, term()}

size/1

size(X1::index()) -> non_neg_integer()

version/0

version() -> binary()


Generated by EDoc