# Beamscope v0.1.2 - Table of Contents

> Compiler-accurate code intelligence for BEAM codebases: chunking and call-graph extraction for Erlang and Elixir via :epp/Code.string_to_quoted, not a generic tree-sitter grammar.

## Pages

- [Beamscope](readme.md)
- [Engineering notes](engineering.md)

## Modules

- [Beamscope](Beamscope.md): Documentation for `Beamscope`.

- [Beamscope.Benchmark.Baseline](Beamscope.Benchmark.Baseline.md): The "no beamscope" comparison point: a real grep-equivalent scan (find
every file that literally contains any of the given `terms`) followed by
a real full read of every matching file — the same methodology used for
`docs/search-benchmark-2026-07.md`'s manual benchmark ("sizes are the
actual bytes returned by each approach"), automated instead of run by
hand. Uses `Beamscope.Chunking.Pipeline.walk_repo/1` for file enumeration,
the same file-discovery logic beamscope itself uses for indexing, so the
baseline stays scoped to what the tool itself considers "the repo's
code."
- [Beamscope.Benchmark.Report](Beamscope.Benchmark.Report.md): Renders one or more `Beamscope.Benchmark.Runner.run/1` results as a
Markdown report matching `docs/search-benchmark-2026-07.md`'s table
format, and writes it to disk with an ISO8601 timestamp in the filename
— so re-running the benchmark never silently overwrites a previous
result.

- [Beamscope.Benchmark.Runner](Beamscope.Benchmark.Runner.md): Orchestrates one repo's benchmark: auto-discovers representative tasks
(`Beamscope.Benchmark.TaskDiscovery`), measures the real "no beamscope"
baseline (`Beamscope.Benchmark.Baseline`) against the real beamscope
`Beamscope.Repo` output for each, counts real tokens
(`Beamscope.Benchmark.Tokenizer`), and times both with `Benchee` for a
repeatable latency comparison — the same methodology used for
`docs/search-benchmark-2026-07.md`'s manual benchmark, automated so it
can be run again and again against any repo.

- [Beamscope.Benchmark.TaskDiscovery](Beamscope.Benchmark.TaskDiscovery.md): Auto-picks representative call-graph/search tasks from a repo's built
call graph, so `mix beamscope.benchmark` doesn't need hardcoded
per-repo function names — the same task shapes used for
`docs/search-benchmark-2026-07.md`'s manual benchmark
(`get_callers`/`get_callees`/`find_call_path`/`search_code`), automated
instead of hand-picked, so the tool works against any repo.

- [Beamscope.Benchmark.Tokenizer](Beamscope.Benchmark.Tokenizer.md): Real BPE token counts, natively in Elixir — no Python/`tiktoken` venv.
- [Beamscope.Callgraph.Extractor](Beamscope.Callgraph.Extractor.md): Extracts function definitions and call edges from a single file using
the same in-process parsers as the chunkers (`:epp` for Erlang,
`Code.string_to_quoted` for Elixir) — not a second tree-sitter pass,
which is what the original `call_extractor.py` uses. This is Phase 0's
core untested hypothesis for the call graph: that a compiler-accurate
parse can drive it too, and — for Erlang specifically — one that sees
through macro expansion in a way tree-sitter structurally cannot, since
`:epp` fully expands macros as part of parsing while tree-sitter only
ever sees the raw `?MACRO(...)` token.
- [Beamscope.Callgraph.Graph](Beamscope.Callgraph.Graph.md): Builds a `libgraph`-backed call graph from `Extractor`'s defs/edges.
- [Beamscope.Callgraph.Pipeline](Beamscope.Callgraph.Pipeline.md): Walks a repo and extracts call-graph defs/edges from every file
concurrently via `Task.Supervisor.async_stream_nolink`, mirroring
`Beamscope.Chunking.Pipeline.chunk_repo/2` (same file discovery,
same concurrency model, same `:telemetry` events under a
`[:beamscope, :callgraph_repo, ...]` prefix). Kept as a separate
pipeline from chunking rather than folded into it, since defs/edges
accumulate differently (into one shared graph) than chunks (into one
flat list).

- [Beamscope.Callgraph.Store](Beamscope.Callgraph.Store.md): Caches built call graphs per repo path so repeated queries (e.g. from MCP
tool calls) don't re-walk and re-parse the whole repo on every call.
- [Beamscope.Chunking.ElixirChunker](Beamscope.Chunking.ElixirChunker.md): Chunks `.ex`/`.exs` files by walking the quoted AST for
`defmodule`/`def`/`defmacro` boundaries, using `Code.string_to_quoted/2`
in-process (no subprocess).
- [Beamscope.Chunking.ErlangChunker](Beamscope.Chunking.ErlangChunker.md): Chunks `.erl`/`.hrl` files by calling `:epp` in-process (no subprocess),
so macros and includes are resolved exactly as the compiler would resolve
them.
- [Beamscope.Chunking.IncludePaths](Beamscope.Chunking.IncludePaths.md): Discovers rebar3 `_build/*/lib` include/ebin directories so `:epp` can
resolve `-include_lib` for third-party deps. Mirrors chunker.py's
`find_include_dirs`/`find_ebin_dirs`/`_find_build_lib_dirs`, which the
original pipeline relies on for every configured repo (see repos.json's
`erlang_include_paths: ["_build/default/lib"]`).

- [Beamscope.Chunking.Pipeline](Beamscope.Chunking.Pipeline.md): Walks a repo and chunks every file concurrently via
`Task.Supervisor.async_stream_nolink`, merging the per-file chunk lists
into one combined list. Each file's parse (`:epp`/`Code.string_to_quoted`)
is a pure, self-contained operation with no shared state, so files are
trivially parallelizable — only the final merge step needs to wait for
every task. Using the supervised, `nolink` variant (rather than plain
`Task.async_stream`) means a file that raises an exception (not just one
that times out) is isolated to that file's `:errors` entry instead of
crashing the caller — which, for callers like `Beamscope.Search.Store`, is
a singleton GenServer caching state for every repo ever queried.
- [Beamscope.Chunking.ProgressReporter](Beamscope.Chunking.ProgressReporter.md): Optional `:telemetry` handler that prints a live-updating progress line
and a polished summary while `Pipeline.chunk_repo/2` runs.
- [Beamscope.Chunking.Support](Beamscope.Chunking.Support.md): Line-extraction, sliding-window, and oversized-chunk-splitting helpers
shared by the per-language chunkers. Mirrors chunker.py's `_slice_lines`,
`chunk_by_lines`, and `_split_oversized_chunks` so chunk boundaries stay
comparable to the original Python pipeline.

- [Beamscope.Chunking.TextChunker](Beamscope.Chunking.TextChunker.md): Shared line-window fallback used by both language chunkers when a source
file cannot be parsed at all. Mirrors chunker.py's `chunk_by_lines`.

- [Beamscope.Embeddings](Beamscope.Embeddings.md): In-process text embeddings via Bumblebee — no external service, no
Ollama/Qdrant, nothing to install or run outside `mix deps.get`.
- [Beamscope.ErlangForms](Beamscope.ErlangForms.md): Shared `:epp` parsing used by both `ErlangChunker` and the call-graph
extractor: parses a file in-process and filters out forms that
originated in an `-include`'d header rather than the file itself.
- [Beamscope.MCP.Protocol](Beamscope.MCP.Protocol.md): Minimal MCP (JSON-RPC 2.0 over HTTP) dispatch: `initialize`, `tools/list`,
`tools/call`. Built directly on Jason-decoded maps rather than a
protocol library — this project only needs a handful of stateless tools,
so a full client/server SDK (session tracking, SSE, batching, capability
negotiation beyond `tools`) is more machinery than the problem calls
for. Batch requests (a JSON array of messages in one body) aren't
supported — each HTTP request is exactly one JSON-RPC message.

- [Beamscope.MCP.Router](Beamscope.MCP.Router.md): HTTP endpoint for the MCP server. A single `POST /mcp` accepting one
JSON-RPC 2.0 message per request, dispatched via `Beamscope.MCP.Protocol`.

- [Beamscope.MCP.Tool](Beamscope.MCP.Tool.md): Behaviour for an MCP tool: a name, description, JSON Schema for its
input, and a call function. Deliberately plain — no macro DSL, no
schema-validation library — since `Beamscope.MCP.Protocol` does the only
validation actually needed (required-key presence) itself.

- [Beamscope.MCP.Tools.FindCallPath](Beamscope.MCP.Tools.FindCallPath.md): MCP tool: find the shortest call path between two functions, if one exists.
- [Beamscope.MCP.Tools.GetCallees](Beamscope.MCP.Tools.GetCallees.md): MCP tool: list every function a given module:function calls.
- [Beamscope.MCP.Tools.GetCallers](Beamscope.MCP.Tools.GetCallers.md): MCP tool: list every function that calls a given module:function.
- [Beamscope.MCP.Tools.SearchCode](Beamscope.MCP.Tools.SearchCode.md): MCP tool: searches a repo for a natural-language or exact-name query,
returning two separate result lists — `exact_matches` (a literal grep for
identifier-like terms in the query, works with no extra dependencies) and
`semantic_matches` (embedding search over the repo's chunks, requires the
optional `bumblebee`/`nx`/`torchx` deps — see `Beamscope.Embeddings`).
They're kept separate rather than blended into one ranking, since an
exact match and a similarity score aren't the same kind of signal — grep
reliably wins on exact-name queries where semantic search is only
probabilistic. If the ML deps aren't installed, `semantic_matches` is
empty and `semantic_search_unavailable` explains why, but `exact_matches`
still works.

- [Beamscope.Repo](Beamscope.Repo.md): Unified per-repo entry point over call-graph navigation and semantic
search — the single interface MCP tools (and any other caller) should
use rather than reaching into `Beamscope.Callgraph.Store` or
`Beamscope.Search.Store` directly. Both stores already cache/persist per
`repo_path` on their own; this module adds no state of its own, just one
API surface over both.

- [Beamscope.Search.LexicalSearch](Beamscope.Search.LexicalSearch.md): Exact/literal-text search over a repo's files for identifier-like terms
pulled out of a natural-language `search_code` query — an in-process grep
(no subprocess), run alongside `Beamscope.Search.Store`'s embedding search
rather than instead of it.
- [Beamscope.Search.Store](Beamscope.Search.Store.md): Per-repo semantic search index: chunks a repo (via
`Beamscope.Chunking.Pipeline`), embeds each chunk (via
`Beamscope.Embeddings`), and persists `{key, vector, metadata}` rows to a
DETS table at `<repo_path>/.beamscope/search.dets` — a build artifact of
the target repo, parallel to `_build/`, meant to be gitignored there.

## Mix Tasks

- [mix beamscope.benchmark](Mix.Tasks.Beamscope.Benchmark.md): Benchmarks beamscope against a real grep/read baseline, for one or more
repos of your choice — self-serve, repeatable, no external Python/tiktoken
dependency.
- [mix beamscope.install](Mix.Tasks.Beamscope.Install.md): Installer for `beamscope`, invoked via `mix igniter.install beamscope`.
- [mix beamscope.mcp](Mix.Tasks.Beamscope.Mcp.md): Starts the Beamscope MCP server over HTTP.

