Arcadic.Vector (Arcadic v0.3.0)

Copy Markdown View Source

ArcadeDB dense vector search — tenant-blind index DDL and nearest-neighbour / hybrid-fusion query builders over ArcadeDB's LSM_VECTOR surface.

Every caller value (query vector, k, ef_search, max_distance) binds as a $param. The only text interpolated into a statement is the index reference "Type[property]", whose two identifiers are Arcadic.Identifier-validated before composition — the sole injection surface, closed by construction (callers pass type/property, never a raw ref). Index metadata values (dimensions, the similarity/encoding/quantization enums) are developer-supplied schema config, validated against integer/allowlist checks before interpolation. Failures carry the invalid SHAPE only, never the offending value (AGENTS.md Critical Rule 3).

Summary

Functions

Creates a dense LSM_VECTOR index (idempotent — IF NOT EXISTS). opts: similarity (:cosine default | :dot_product | :euclidean), encoding (:float32 | :int8), quantization (:none | :int8 | :binary | :product), max_connections (default 16), beam_width (default 100). Unknown opt keys are rejected value-free (the server silently accepts unknown METADATA keys).

Creates a dense vector index, raising on error.

Creates an LSM_SPARSE_VECTOR index over a (tokens, weights) property pair. opts: dimensions (pos_integer, optional) and modifier (:none default | :idf). Both are omitted from the DDL when not given (server defaults apply).

Drops a dense vector index (idempotent — IF EXISTS).

Drops a dense vector index, raising on error.

Drops a sparse vector index (idempotent — IF EXISTS).

Drops a sparse vector index, raising on error.

Runs a hybrid fusion over dense neighbour subqueries. neighbor_specs is a non-empty list of {type, property, query_vector, k}, each built as a validated vector.neighbors subquery with distinct indexed params. opts: fusion (:rrf default | :dbsf | :linear), weights (list of numbers), k (pos_integer), filter (non-empty list of #<bucket>:<pos> candidate RID strings — a SHARED candidate set threaded into every subquery), group_by (property name to group by; validated for identifier shape), group_size (pos_integer — max results per group). group_by/group_size apply to the FUSED output (the outer fuse object); all bind as params.

Runs a hybrid fusion, returning rows or raising.

Builds the ArcadeDB index reference "Type[property]", validating both identifiers.

Runs a dense nearest-neighbour search, returning rows ranked closest-first (each carries the vertex's top-level fields plus distance). query_vector and k bind as params. opts: ef_search (pos_integer), max_distance (number), filter (non-empty list of #<bucket>:<pos> candidate RID strings — restricts the search to that candidate set), group_by (property name to group results by; validated for identifier shape), group_size (pos_integer — max results per group). All bind as params inside the query-options object.

Runs a dense nearest-neighbour search, returning rows or raising.

Builds the sparse index ref "Type[tokens,weights]", validating all three identifiers.

Runs a sparse (learned-sparse / BM25-style) nearest-neighbour search over an LSM_SPARSE_VECTOR index, returning rows ranked by a top-level score (higher = better; there is no distance). query_tokens/query_weights/k bind as params. opts: filter (candidate RID set), group_by, group_sizeef_search/max_distance are not accepted (rejected client-side value-free). See create_sparse_index/5 for the index coverage caveat.

Functions

create_dense_index(conn, type, property, dimensions, opts \\ [])

@spec create_dense_index(
  Arcadic.Conn.t(),
  String.t(),
  String.t(),
  pos_integer(),
  keyword()
) ::
  :ok | {:error, atom() | Exception.t()}

Creates a dense LSM_VECTOR index (idempotent — IF NOT EXISTS). opts: similarity (:cosine default | :dot_product | :euclidean), encoding (:float32 | :int8), quantization (:none | :int8 | :binary | :product), max_connections (default 16), beam_width (default 100). Unknown opt keys are rejected value-free (the server silently accepts unknown METADATA keys).

create_dense_index!(conn, type, property, dimensions, opts \\ [])

@spec create_dense_index!(
  Arcadic.Conn.t(),
  String.t(),
  String.t(),
  pos_integer(),
  keyword()
) :: :ok

Creates a dense vector index, raising on error.

create_sparse_index(conn, type, tokens_property, weights_property, opts \\ [])

@spec create_sparse_index(
  Arcadic.Conn.t(),
  String.t(),
  String.t(),
  String.t(),
  keyword()
) ::
  :ok | {:error, atom() | Exception.t()}

Creates an LSM_SPARSE_VECTOR index over a (tokens, weights) property pair. opts: dimensions (pos_integer, optional) and modifier (:none default | :idf). Both are omitted from the DDL when not given (server defaults apply).

Coverage caveat (ArcadeDB semantic): a sparse index does NOT index rows that existed BEFORE it was created — only rows inserted or updated afterwards are searchable (the DDL's reported index count is misleading, and REBUILD INDEX throws on a sparse index). Create the index before loading, or re-touch pre-existing rows. When the DDL reports pre-existing rows, a value-free [:arcadic, :vector, :sparse_index_preexisting] telemetry event is emitted.

create_sparse_index!(conn, type, tokens_property, weights_property, opts \\ [])

@spec create_sparse_index!(
  Arcadic.Conn.t(),
  String.t(),
  String.t(),
  String.t(),
  keyword()
) :: :ok

Creates a sparse vector index, raising on error.

drop_dense_index(conn, type, property)

@spec drop_dense_index(Arcadic.Conn.t(), String.t(), String.t()) ::
  :ok | {:error, atom() | Exception.t()}

Drops a dense vector index (idempotent — IF EXISTS).

drop_dense_index!(conn, type, property)

@spec drop_dense_index!(Arcadic.Conn.t(), String.t(), String.t()) :: :ok

Drops a dense vector index, raising on error.

drop_sparse_index(conn, type, tokens_property, weights_property)

@spec drop_sparse_index(Arcadic.Conn.t(), String.t(), String.t(), String.t()) ::
  :ok | {:error, atom() | Exception.t()}

Drops a sparse vector index (idempotent — IF EXISTS).

drop_sparse_index!(conn, type, tokens_property, weights_property)

@spec drop_sparse_index!(Arcadic.Conn.t(), String.t(), String.t(), String.t()) :: :ok

Drops a sparse vector index, raising on error.

fuse(conn, neighbor_specs, opts \\ [])

@spec fuse(
  Arcadic.Conn.t(),
  [{String.t(), String.t(), [number()], pos_integer()}],
  keyword()
) ::
  {:ok, [map()]} | {:error, atom() | Exception.t()}

Runs a hybrid fusion over dense neighbour subqueries. neighbor_specs is a non-empty list of {type, property, query_vector, k}, each built as a validated vector.neighbors subquery with distinct indexed params. opts: fusion (:rrf default | :dbsf | :linear), weights (list of numbers), k (pos_integer), filter (non-empty list of #<bucket>:<pos> candidate RID strings — a SHARED candidate set threaded into every subquery), group_by (property name to group by; validated for identifier shape), group_size (pos_integer — max results per group). group_by/group_size apply to the FUSED output (the outer fuse object); all bind as params.

Fused rows are ranked by ArcadeDB's fusion score (higher = better) rather than the distance that neighbors/6 returns.

neighbor_specs and weights are trusted developer-supplied config: the emitted statement grows linearly with their length (no cap), matching arcadic's no-statement-size-limit posture elsewhere.

fuse!(conn, neighbor_specs, opts \\ [])

@spec fuse!(
  Arcadic.Conn.t(),
  [{String.t(), String.t(), [number()], pos_integer()}],
  keyword()
) :: [
  map()
]

Runs a hybrid fusion, returning rows or raising.

index_ref(type, property)

@spec index_ref(String.t(), String.t()) ::
  {:ok, String.t()} | {:error, :invalid_identifier}

Builds the ArcadeDB index reference "Type[property]", validating both identifiers.

neighbors(conn, type, property, query_vector, k, opts \\ [])

@spec neighbors(
  Arcadic.Conn.t(),
  String.t(),
  String.t(),
  [number()],
  pos_integer(),
  keyword()
) ::
  {:ok, [map()]} | {:error, atom() | Exception.t()}

Runs a dense nearest-neighbour search, returning rows ranked closest-first (each carries the vertex's top-level fields plus distance). query_vector and k bind as params. opts: ef_search (pos_integer), max_distance (number), filter (non-empty list of #<bucket>:<pos> candidate RID strings — restricts the search to that candidate set), group_by (property name to group results by; validated for identifier shape), group_size (pos_integer — max results per group). All bind as params inside the query-options object.

distance (and therefore max_distance) semantics depend on the index's similarity: COSINE yields 0..1 ascending (0 = identical); DOT_PRODUCT yields NEGATIVE values (≈ -1 identical, less-negative = farther), so a small positive max_distance filters nothing; EUCLIDEAN differs again. Choose thresholds per the index's similarity.

neighbors!(conn, type, property, query_vector, k, opts \\ [])

@spec neighbors!(
  Arcadic.Conn.t(),
  String.t(),
  String.t(),
  [number()],
  pos_integer(),
  keyword()
) :: [
  map()
]

Runs a dense nearest-neighbour search, returning rows or raising.

sparse_index_ref(type, tokens_property, weights_property)

@spec sparse_index_ref(String.t(), String.t(), String.t()) ::
  {:ok, String.t()} | {:error, :invalid_identifier}

Builds the sparse index ref "Type[tokens,weights]", validating all three identifiers.

sparse_neighbors(conn, type, tokens_property, weights_property, query_tokens, query_weights, k, opts \\ [])

@spec sparse_neighbors(
  Arcadic.Conn.t(),
  String.t(),
  String.t(),
  String.t(),
  [integer()],
  [number()],
  pos_integer(),
  keyword()
) :: {:ok, [map()]} | {:error, atom() | Exception.t()}

Runs a sparse (learned-sparse / BM25-style) nearest-neighbour search over an LSM_SPARSE_VECTOR index, returning rows ranked by a top-level score (higher = better; there is no distance). query_tokens/query_weights/k bind as params. opts: filter (candidate RID set), group_by, group_sizeef_search/max_distance are not accepted (rejected client-side value-free). See create_sparse_index/5 for the index coverage caveat.

sparse_neighbors!(conn, type, tokens_property, weights_property, query_tokens, query_weights, k, opts \\ [])

@spec sparse_neighbors!(
  Arcadic.Conn.t(),
  String.t(),
  String.t(),
  String.t(),
  [integer()],
  [number()],
  pos_integer(),
  keyword()
) :: [map()]

Runs a sparse nearest-neighbour search, returning rows or raising.