Harlock.Bench (harlock v0.7.0)

Frame-timing harness for element trees.

Two costs make up a frame: turning an element tree into a Frame, and diffing two frames into the ANSI bytes that transition between them. This measures both, over a set of scenarios that can be re-run to catch regressions.

It is deliberately public rather than a dev-only script, because the more useful question is usually about your view rather than Harlock's:

iex> Harlock.Bench.render(my_view(model), rows: 50, cols: 120) |> Harlock.Bench.format()

Reading the numbers

Results report percentiles, not just a mean. A frame budget is about the slow frames — a 60 Hz target has ~16 000µs to spend, and a p99 that blows it is visible stutter even when the mean looks fine. Means also hide the GC pauses and scheduler hops that percentiles expose.

Timings come from :timer.tc/1, so they include whatever else the VM was doing. Comparisons between runs on one machine are meaningful; absolute numbers across machines are not.

Rendering here is pure — no terminal, no IO, no Writer. Turning the diff's iodata into bytes and writing it is a separate cost this does not measure.

Summary

Types

stats()

Microsecond timings for one measured operation.

Functions

diff(before_el, after_el, opts \\ [])

Time diffing the frames produced by two element trees.

format(stats)

Format stats/0 or a list of {name, stats} pairs as an aligned table.

measure(fun, opts \\ [])

Time a zero-arity function.

render(element, opts \\ [])

Time rendering an element tree into a frame.

render_varying(build, opts \\ [])

Time rendering a different tree on each sample.

run(opts \\ [])

Render every scenario and return {name, stats} pairs.

scenario(name, opts \\ [])

Build one scenario by name.

scenarios(opts \\ [])

Canonical scenarios, as {name, element} pairs.

Types

stats()

@type stats() :: %{
  samples: pos_integer(),
  min: non_neg_integer(),
  p50: non_neg_integer(),
  p95: non_neg_integer(),
  p99: non_neg_integer(),
  max: non_neg_integer(),
  mean: non_neg_integer()
}

Microsecond timings for one measured operation.

Functions

diff(before_el, after_el, opts \\ [])

@spec diff(Harlock.Element.t(), Harlock.Element.t(), keyword()) :: stats()

Time diffing the frames produced by two element trees.

Both trees are rendered once up front, so this measures only the diff. Pass the same tree twice to measure the no-change path, which is the common case for a dirty-flag runtime and should be cheap.

format(stats)

@spec format(stats() | [{atom(), stats()}]) :: String.t()

Format stats/0 or a list of {name, stats} pairs as an aligned table.

measure(fun, opts \\ [])

@spec measure(
  (-> any()),
  keyword()
) :: stats()

Time a zero-arity function.

Options: :samples (default 100) and :warmup (default 20). Warmup runs are discarded — the first few passes over cold code and caches are not representative of a steady-state frame.

render(element, opts \\ [])

@spec render(
  Harlock.Element.t(),
  keyword()
) :: stats()

Time rendering an element tree into a frame.

Options: :rows (default 200), :cols (default 80), plus anything measure/2 takes.

render_varying(build, opts \\ [])

@spec render_varying(
  (pos_integer() -> Harlock.Element.t()),
  keyword()
) :: stats()

Time rendering a different tree on each sample.

Necessary wherever a cache sits behind the thing being measured. render/2 hands the same tree over and over, so any memo keyed on content hits from the second sample onward and the result describes a steady-state redraw of unchanged content. That is a real case — but it is not the one a user typing into a textarea experiences, where every keystroke changes the value and misses.

build receives the sample number and returns the tree for it. It is called outside the timed section, and each tree becomes garbage as soon as its sample finishes — holding a list of large variants alive instead adds GC pressure that inflates every reading. That mistake overstated this measurement by more than a factor of two the first time it was written.

Bench.render_varying(&Bench.scenario(:textarea_wrapped, n: 200, edit: &1), samples: 50)

run(opts \\ [])

@spec run(keyword()) :: [{atom(), stats()}]

Render every scenario and return {name, stats} pairs.

:textarea_wrapped is the one to watch: every render rewraps the whole value, which is the cost the roadmap's incremental-rewrap work exists to remove. A baseline here is what makes that change measurable rather than assumed.

scenario(name, opts \\ [])

@spec scenario(
  atom(),
  keyword()
) :: Harlock.Element.t()

Build one scenario by name.

:n scales the content. :edit perturbs it, which is what makes each tree distinct for render_varying/2 — a content-keyed cache has to miss for the measurement to describe editing rather than redrawing.

scenarios(opts \\ [])

@spec scenarios(keyword()) :: [{atom(), Harlock.Element.t()}]

Canonical scenarios, as {name, element} pairs.

These exist so a baseline is reproducible rather than ad hoc. :n scales the data-heavy ones (default 200, matching the default row count so the content slightly overflows the screen — the interesting case, since it exercises clipping).