Measures the heap cost of the PtcRunner embedding units and compares them
against bench/baselines/heap.json.
mix bench.heap # measure, compare, fail on regression
mix bench.heap --write-baseline # re-record the committed baseline
mix bench.heap --report # measure and print, never failmix bench.check calls report/1 to print the same table informationally
alongside the eval-reduction gate. This task is the gating entry point; run
it deliberately, at release time or from the scheduled Soak workflow.
Rows
| row | what it sizes |
|---|---|
floor_no_mix_idle | resident cost of a started :ptc_runner with no Mix on the path |
floor_no_mix_warm | the same child after one Lisp.run/2, so lazy loading has happened |
lisp_trivial | the cheapest possible Lisp.run/2 unit |
lisp_collection_hofs | allocation-heavy builtins |
lisp_large_binary | refc-binary construction and its residual |
compile_bundle_1 | per-artifact resident cost of a compiled bundle |
compile_bundle_20 | whether bundle cost scales linearly in components |
kernel_run | the real embedding unit |
kernel_run_capabilities_events | the same unit with a capability and an event sink |
concurrency_N | heap per concurrent Kernel.run, for N in 1/8/32/64/128 |
Metrics
unit_bytes— the workload's own heap at return, before any GC. For thelisp_*rows this isstep.usage.memory_bytes, the sandbox child'sProcess.info(self(), :memory)(sandbox.ex). For thecompile_*andkernel_*rows it is the same reading taken in a dedicated driver process.PtcRunner.Kernel.run/2spawns its own sandbox children, so thekernel_*driver figure excludes them —concurrency_*is the row that sees the whole system.retained_bytes— the retained size of the durable artifact the row produces, from PtcRunner.Lisp.RetainedSize, ornullwhen that module answers:oversized.residual_bytes_per_iter— repeated-batch residual. K equal batches of N iterations with a system-wide GC at each boundary; the reported rate is(endpoint_K - endpoint_1) / ((K - 1) * N)over:erlang.memory(:total). Batch 1 is deliberately excluded: it carries lazy module loading and first-use interning, and a single before/after delta cannot tell that one-shot cost apart from a linear leak. At these iteration counts the metric has no resolution below roughly half a kilobyte per iteration — see@metric_floors. It is here to be read as a trend; the soak suite is where residual growth is hard-asserted, because only there is the workload repeated enough for a slope to outrun the VM's own drift.worker_heap_bytes_total/worker_heap_bytes_median—concurrency_*only. Each concurrent worker reports its own heap at return, so these are deterministic and are what this row gates on. Like thekernel_*rows they exclude the sandbox children.peak_total_bytes/peak_bytes_per_run—concurrency_*only, from a sampling process reading:erlang.memory(:total). This is the only sample that sees the sandbox children, and it is diagnostic, never compared::erlang.memory/0walks every allocator, so the sampler lands ~10 reads on a sub-millisecond run and the figure moved by more than 2x between runs of the unchanged workload.samples_observedrecords that coverage.
Every figure is a heap figure, not RSS. Allocator carrier fragmentation and resident-set growth are invisible here by construction; the plan puts both in the diagnostic tier, and neither is sampled at all by this task.
Summary
Functions
Measures every row and prints the comparison table without failing.
Functions
@spec report(keyword()) :: :ok
Measures every row and prints the comparison table without failing.
This is what mix bench.check calls: the heap table is informational there,
because a re-baseline reflex is the failure mode a byte gate invites, and the
hard assertions live in the soak suite where the workload is repeated enough
for a slope to mean something.