API Reference Beamscope v#0.1.0

Documentation for Beamscope.

The "no beamscope" comparison point: a real grep-equivalent scan (find every file that literally contains any of the given terms) followed by a real full read of every matching file — the same methodology used for docs/search-benchmark-2026-07.md's manual benchmark ("sizes are the actual bytes returned by each approach"), automated instead of run by hand. Uses Beamscope.Chunking.Pipeline.walk_repo/1 for file enumeration, the same file-discovery logic beamscope itself uses for indexing, so the baseline stays scoped to what the tool itself considers "the repo's code."

Renders one or more Beamscope.Benchmark.Runner.run/1 results as a Markdown report matching docs/search-benchmark-2026-07.md's table format, and writes it to disk with an ISO8601 timestamp in the filename — so re-running the benchmark never silently overwrites a previous result.

Orchestrates one repo's benchmark: auto-discovers representative tasks (Beamscope.Benchmark.TaskDiscovery), measures the real "no beamscope" baseline (Beamscope.Benchmark.Baseline) against the real beamscope Beamscope.Repo output for each, counts real tokens (Beamscope.Benchmark.Tokenizer), and times both with Benchee for a repeatable latency comparison — the same methodology used for docs/search-benchmark-2026-07.md's manual benchmark, automated so it can be run again and again against any repo.

Auto-picks representative call-graph/search tasks from a repo's built call graph, so mix beamscope.benchmark doesn't need hardcoded per-repo function names — the same task shapes used for docs/search-benchmark-2026-07.md's manual benchmark (get_callers/get_callees/find_call_path/search_code), automated instead of hand-picked, so the tool works against any repo.

Real BPE token counts, natively in Elixir — no Python/tiktoken venv.

Extracts function definitions and call edges from a single file using the same in-process parsers as the chunkers (:epp for Erlang, Code.string_to_quoted for Elixir) — not a second tree-sitter pass, which is what the original call_extractor.py uses. This is Phase 0's core untested hypothesis for the call graph: that a compiler-accurate parse can drive it too, and — for Erlang specifically — one that sees through macro expansion in a way tree-sitter structurally cannot, since :epp fully expands macros as part of parsing while tree-sitter only ever sees the raw ?MACRO(...) token.

Builds a libgraph-backed call graph from Extractor's defs/edges.

Walks a repo and extracts call-graph defs/edges from every file concurrently via Task.Supervisor.async_stream_nolink, mirroring Beamscope.Chunking.Pipeline.chunk_repo/2 (same file discovery, same concurrency model, same :telemetry events under a [:beamscope, :callgraph_repo, ...] prefix). Kept as a separate pipeline from chunking rather than folded into it, since defs/edges accumulate differently (into one shared graph) than chunks (into one flat list).

Caches built call graphs per repo path so repeated queries (e.g. from MCP tool calls) don't re-walk and re-parse the whole repo on every call.

Chunks .ex/.exs files by walking the quoted AST for defmodule/def/defmacro boundaries, using Code.string_to_quoted/2 in-process (no subprocess).

Chunks .erl/.hrl files by calling :epp in-process (no subprocess), so macros and includes are resolved exactly as the compiler would resolve them.

Discovers rebar3 _build/*/lib include/ebin directories so :epp can resolve -include_lib for third-party deps. Mirrors chunker.py's find_include_dirs/find_ebin_dirs/_find_build_lib_dirs, which the original pipeline relies on for every configured repo (see repos.json's erlang_include_paths: ["_build/default/lib"]).

Walks a repo and chunks every file concurrently via Task.Supervisor.async_stream_nolink, merging the per-file chunk lists into one combined list. Each file's parse (:epp/Code.string_to_quoted) is a pure, self-contained operation with no shared state, so files are trivially parallelizable — only the final merge step needs to wait for every task. Using the supervised, nolink variant (rather than plain Task.async_stream) means a file that raises an exception (not just one that times out) is isolated to that file's :errors entry instead of crashing the caller — which, for callers like Beamscope.Search.Store, is a singleton GenServer caching state for every repo ever queried.

Optional :telemetry handler that prints a live-updating progress line and a polished summary while Pipeline.chunk_repo/2 runs.

Line-extraction, sliding-window, and oversized-chunk-splitting helpers shared by the per-language chunkers. Mirrors chunker.py's _slice_lines, chunk_by_lines, and _split_oversized_chunks so chunk boundaries stay comparable to the original Python pipeline.

Shared line-window fallback used by both language chunkers when a source file cannot be parsed at all. Mirrors chunker.py's chunk_by_lines.

In-process text embeddings via Bumblebee — no external service, no Ollama/Qdrant, nothing to install or run outside mix deps.get.

Shared :epp parsing used by both ErlangChunker and the call-graph extractor: parses a file in-process and filters out forms that originated in an -include'd header rather than the file itself.

Minimal MCP (JSON-RPC 2.0 over HTTP) dispatch: initialize, tools/list, tools/call. Built directly on Jason-decoded maps rather than a protocol library — this project only needs a handful of stateless tools, so a full client/server SDK (session tracking, SSE, batching, capability negotiation beyond tools) is more machinery than the problem calls for. Batch requests (a JSON array of messages in one body) aren't supported — each HTTP request is exactly one JSON-RPC message.

HTTP endpoint for the MCP server. A single POST /mcp accepting one JSON-RPC 2.0 message per request, dispatched via Beamscope.MCP.Protocol.

Behaviour for an MCP tool: a name, description, JSON Schema for its input, and a call function. Deliberately plain — no macro DSL, no schema-validation library — since Beamscope.MCP.Protocol does the only validation actually needed (required-key presence) itself.

MCP tool: find the shortest call path between two functions, if one exists.

MCP tool: list every function a given module:function calls.

MCP tool: list every function that calls a given module:function.

MCP tool: searches a repo for a natural-language or exact-name query, returning two separate result lists — exact_matches (a literal grep for identifier-like terms in the query, works with no extra dependencies) and semantic_matches (embedding search over the repo's chunks, requires the optional bumblebee/nx/torchx deps — see Beamscope.Embeddings). They're kept separate rather than blended into one ranking, since an exact match and a similarity score aren't the same kind of signal — grep reliably wins on exact-name queries where semantic search is only probabilistic. If the ML deps aren't installed, semantic_matches is empty and semantic_search_unavailable explains why, but exact_matches still works.

Unified per-repo entry point over call-graph navigation and semantic search — the single interface MCP tools (and any other caller) should use rather than reaching into Beamscope.Callgraph.Store or Beamscope.Search.Store directly. Both stores already cache/persist per repo_path on their own; this module adds no state of its own, just one API surface over both.

Exact/literal-text search over a repo's files for identifier-like terms pulled out of a natural-language search_code query — an in-process grep (no subprocess), run alongside Beamscope.Search.Store's embedding search rather than instead of it.

Per-repo semantic search index: chunks a repo (via Beamscope.Chunking.Pipeline), embeds each chunk (via Beamscope.Embeddings), and persists {key, vector, metadata} rows to a DETS table at <repo_path>/.beamscope/search.dets — a build artifact of the target repo, parallel to _build/, meant to be gitignored there.

Benchmarks beamscope against a real grep/read baseline, for one or more repos of your choice — self-serve, repeatable, no external Python/tiktoken dependency.

Installer for beamscope, invoked via mix igniter.install beamscope.

Starts the Beamscope MCP server over HTTP.