Beamscope.Search.Store (Beamscope v0.1.2)

Copy Markdown View Source

Per-repo semantic search index: chunks a repo (via Beamscope.Chunking.Pipeline), embeds each chunk (via Beamscope.Embeddings), and persists {key, vector, metadata} rows to a DETS table at <repo_path>/.beamscope/search.dets — a build artifact of the target repo, parallel to _build/, meant to be gitignored there.

DETS is the durable store; like Beamscope.Callgraph.Store, the built index also lives in this GenServer's in-memory state as a plain list, so repeated searches are in-memory scans rather than repeated disk reads. DETS exists so a server restart doesn't require re-embedding an entire repo — get_or_build/2 opens an existing table if one is already on disk instead of rebuilding from scratch.

A fresh build writes to a search.dets.tmp.<unique> file, only renamed to the real search.dets path after a clean :dets.close/1 — so a process crash mid-build (e.g. kill -9) leaves only the temp file behind rather than leaving the real file "not properly closed" (which triggers DETS's automatic repair-on-reopen, and can reset it to empty).

Search is brute-force cosine similarity over the in-memory vector list. Realistic scale here is tens of thousands of vectors per repo, well within what a linear scan handles in well under a second — no ANN index needed.

Returns {:error, :embeddings_not_available} from any operation that needs to embed text if the optional bumblebee/nx/torchx deps (see Beamscope.Embeddings) aren't installed.

Summary

Functions

Returns a specification to start this module under a supervisor.

Ensures repo_path has a built index, opening an existing on-disk DETS table if present or building fresh otherwise. Returns :ok, or {:error, :embeddings_not_available} if the optional ML deps aren't installed.

Returns whether repo_path has a cached index, without building one.

Forces a rebuild of the index for repo_path, discarding any cached or on-disk version.

Embeds query and returns the top-K most similar chunks (default 10, override via :limit) for repo_path, building the index first if necessary. Each result is %{file_path:, symbol:, start_line:, end_line:, kind:, score:}.

Types

repo_path()

@type repo_path() :: String.t()

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

get_or_build(repo_path, opts \\ [])

@spec get_or_build(
  repo_path(),
  keyword()
) :: :ok | {:error, term()}

Ensures repo_path has a built index, opening an existing on-disk DETS table if present or building fresh otherwise. Returns :ok, or {:error, :embeddings_not_available} if the optional ML deps aren't installed.

indexed?(repo_path)

@spec indexed?(repo_path()) :: boolean()

Returns whether repo_path has a cached index, without building one.

reindex(repo_path, opts \\ [])

@spec reindex(
  repo_path(),
  keyword()
) :: :ok | {:error, term()}

Forces a rebuild of the index for repo_path, discarding any cached or on-disk version.

search(repo_path, query, opts \\ [])

@spec search(repo_path(), String.t(), keyword()) :: {:ok, [map()]} | {:error, term()}

Embeds query and returns the top-K most similar chunks (default 10, override via :limit) for repo_path, building the index first if necessary. Each result is %{file_path:, symbol:, start_line:, end_line:, kind:, score:}.

start_link(opts \\ [])

@spec start_link(keyword()) :: GenServer.on_start()