PostgreSQL pgvector-backed vector store.

This is the default vector store backend, using the existing Arcana schema with pgvector extension for similarity search.

Breaking name change

In v1.7 the hybrid-search opts :semantic_weight and :fulltext_weight were renamed to :vector_weight and :keyword_weight. The old names are no longer accepted: callers passing them get a Logger.warning and the default 0.5 weight.

Configuration

config :arcana, vector_store: :pgvector  # default

Tuning recall with :hnsw_ef_search

An HNSW index scan considers hnsw.ef_search candidates and only then applies the query's filters, so a search scoped to a collection or a source can come back with fewer rows than exist - occasionally none. pgvector's default is 40. Raising it finds more of the true nearest neighbours, at the cost of more work per query:

Arcana.search("question", collections: ["docs"], hnsw_ef_search: 200)

It can also be a global search default:

config :arcana, search: [hnsw_ef_search: 200]

Three things worth knowing. It only applies when the planner actually chooses an index scan - on a small table a sequential scan is exact and the setting is irrelevant. Because hnsw.ef_search is a GUC, applying it means running the search inside a transaction; if you already have one open, the setting stays in effect for the rest of your transaction rather than just the search.

And it costs a round-trip. Setting it adds a transaction and one set_config query per search, and Arcana.search/2 searches each collection separately - so a query over three collections pays it three times, and as a global default it applies to every search. That is deliberate rather than hoisted up to wrap the whole retrieval: retrieval also embeds the query, and holding a database transaction open across a call to an embedding service is worse than an extra round-trip. Set it per call on the searches that need the recall, rather than globally, if that matters to you.

Applies to the chunk search in :vector and :hybrid modes. :keyword never touches the vector index, so it ignores the option.

It does not reach the graph paths. Arcana.Graph.GraphStore.Ecto's entity search runs its own filtered query against arcana_graph_entities' HNSW index, so with graph: true (or Arcana.ask/2's graph context) the chunk search honours the option while the entity match can still under-return. Extending it there is a separate change.

Notes

This backend works with the existing arcana_chunks and arcana_documents tables. The collection parameter maps to the document's collection_id.

For simpler use cases without the full document schema, consider the :memory backend.