Beamscope.Benchmark.Baseline (Beamscope v0.1.0)

Copy Markdown View Source

The "no beamscope" comparison point: a real grep-equivalent scan (find every file that literally contains any of the given terms) followed by a real full read of every matching file — the same methodology used for docs/search-benchmark-2026-07.md's manual benchmark ("sizes are the actual bytes returned by each approach"), automated instead of run by hand. Uses Beamscope.Chunking.Pipeline.walk_repo/1 for file enumeration, the same file-discovery logic beamscope itself uses for indexing, so the baseline stays scoped to what the tool itself considers "the repo's code."

Reading is capped at @max_bytes total. Real incident that motivated this: on a large monorepo, an auto-discovered term that happened to be extremely common (a short, ubiquitous function name — see Beamscope.Benchmark.TaskDiscovery's "?" unresolved-dispatch exclusion, which this cap backstops for any other equally-common real name) matched nearly the entire repo; accumulating and joining that much content once is already expensive, and Beamscope.Benchmark.Runner re-invokes this function on every Benchee timing iteration — repeated multi-hundred-MB allocations exhausted memory within seconds. capped?: true signals the reported bytes/tokens are a lower bound, not the true full count, when the limit is hit — this measures "grep would return an enormous, most-of-the-repo result," which is itself an honest finding, not a number to hide.

Summary

Functions

Real bytes of every file under repo_path that literally contains any of terms, capped at max_bytes total (default @max_bytes, see moduledoc — overridable mainly so tests can exercise the cap without a multi-megabyte fixture). Returns %{bytes: 0, text: "", capped?: false} for an empty term list — there's nothing to grep for (the honest baseline cost for a purely conceptual query with no exact name to search).

Functions

measure(repo_path, terms, max_bytes \\ 10_000_000)

@spec measure(String.t(), [String.t()], pos_integer()) :: %{
  bytes: non_neg_integer(),
  text: String.t(),
  capped?: boolean()
}

Real bytes of every file under repo_path that literally contains any of terms, capped at max_bytes total (default @max_bytes, see moduledoc — overridable mainly so tests can exercise the cap without a multi-megabyte fixture). Returns %{bytes: 0, text: "", capped?: false} for an empty term list — there's nothing to grep for (the honest baseline cost for a purely conceptual query with no exact name to search).