Tuning a worker host

This guide is for one symptom: your requests are slower than your guest is, and the time is going somewhere you cannot name. Reach for it when a request costs tens of milliseconds and the same call, measured on its own, costs single digits. Most of the time the answer is garbage collection in the process the kernel spawns per request, and the fix is one worker option. The rest of the time it is the start rather than the request, which is a different guide.

Every number here was measured on this project; test/audit/PERF.md is where each one lives, and bench/paths/README.md is the protocol they were taken under.

Find out where the time goes

Do this before changing any setting. Check the load average first, take minimums rather than means, and never compare a number from one run against a number from another.

%% one process's own allocation and collection time
allocwords:measure(fun() -> script_worker:run(W, Req) end).

%% the floors and ceilings a process actually got
erlang:process_info(Pid, garbage_collection).

Do not use erlang:statistics(garbage_collection) for this. It counts the whole node. bench/paths/allocwords.erl exists because that counter reported no change while one process's collections fell 51x, which sent an investigation down the wrong path for an afternoon.

To compare settings, run them in one emulator, interleaved, with the order reversed on alternate rounds:

erl -noshell -pa _build/test/lib/wasm/ebin -pa _build/test/lib/wasm/examples \
    -pa bench/paths -run workerbench main floors qjs_reactor metered 10 0 100000 200000

That is the only comparison worth making on a machine that has other work on it. Run the same arm against itself first: if the two halves of a null experiment differ by more than a few per cent, the box is too busy to measure on at all.

Give the request runner a heap floor

script_worker:start_link(my_adapter, #{root => scratch,
                                       runner_min_heap_words => 200_000}).

A request runner holds almost nothing on its own heap. The module is a cache handle, the memories are atomics pages, a restored image's contents are reference-counted binaries. The collector sizes a heap from the live set, so it gives the runner the emulator's default 233 words and then collects through the request dozens of times while the guest allocates hundreds of millions.

Sweeping QuickJS, ten requests per floor, all arms in one emulator:

The knee is sharp and it plateaus. Past it a floor starts costing again, which is why a sweep has to go beyond the knee rather than stop at the first improvement. CPython at 1,000,000, 2,000,000 and 4,000,000 words is 124, 124 and 145 ms, with the collections still falling, 43 to 29 to 19, and the time spent in them rising, 8.3 to 12.6 ms. A heap too large for its live set means each collection walks more.

Find your own. The right value is a property of the guest, not of this runtime. The three measured here want a 5x spread:

Sweep, take the knee, and stop. Two rules that came from getting it wrong:

• Put the off setting in the sweep. Without it there is nothing in the run to say the floors worked at all, which is how one CPython sweep here came back flat and had to be thrown away.
• Go past the knee. Otherwise you cannot tell a plateau from a peak, and the number you pick may be on the far side of it.

Two things to know before you set it:

• The emulator rounds the number up to a heap-size class, and the jump is large: 200,000 words becomes 318,187, and 1,000 becomes 1,598. That is 2.4 MiB of ballast per concurrent runner, and it does not cost you memory on balance: see the scaling section below.
• The floor must fit under this worker's max_heap_words with room for that rounding. One that does not is refused with a warning and the process gets no floor, because min_heap_size above max_heap_size is a kill at spawn.
• That check is not the whole of it. It catches a floor too large to start under; it cannot catch a floor that starts fine and then leaves too little headroom under the ceiling for the work itself. max_heap_words bounds the peak and a floor raises the baseline the peak is measured from, so raise the two together. CPython captures at the 16 M words its adapter asks for, and with a 2 M capture floor it dies about three times in four. When that happens the error names max_heap_words and the floor rather than only saying killed.

script_worker:runner_heap_words/2 answers what a given pair of options and limits resolves to, so you can check a configuration without starting a worker.

Give the capture a floor as well

script_worker:start_link(my_adapter, #{root => scratch,
                                       capture_min_heap_words => 2_000_000}).

The same mechanism on the process that runs a snapshot capture, and on a guest that takes a long time to start it is worth more than anything else in this guide. A CPython worker start, interleaved, snapshot_dir unset so every arm really captures:

That is the same effect as the request floor, on a process whose live set is small for the same reason, and it is larger because the work is longer. It only applies where a capture happens: a worker reading its image from snapshot_dir pays none of this, and neither does an adapter that declares no snapshot capability.

Do not reach for +hms first

The emulator's own heap settings, with the defaults they have here:

Three notes, each one a wrong knob that is easy to reach for:

• +hms is the node, not the runner. It sizes the guardian, the reaper's children, every process this runtime spawns and everything else in your release. runner_min_heap_words is the same idea scoped to the one process that needs it.
• +hmbs is part of why the symptom exists. The collector derives that threshold from the live set too, so a runner holding nothing keeps the default 46,422 and crosses it constantly.
• +hmaxib is false, which is the emulator's half of what wasm_limits says in prose: linear memory is off-heap and no heap bound can see it.

Read a process's values back with erlang:process_info(Pid, garbage_collection). Read the node's with erlang:system_info(min_heap_size), which answers the tuple {min_heap_size, 233} rather than an integer.

What the runtime already sizes for you

Do not set these again or fight them:

• wasm:compile/1 floors its own heap at two words per input byte. That is what takes QuickJS from 244 ms to 55.
• max_heap_words in a limits map, applied at spawn_opt by whoever owns the instance. See the worker guide.
• compile_max_heap_words bounds a compiler process. See the compiled tier guide.

When the cost is the start, not the request

None of the above helps a guest that takes ninety seconds to come up and a third of a second to answer. Two different settings do:

• snapshot_dir files an initialized image, which takes a CPython worker start from 104 s to 998 ms. See the snapshots guide.
• code_cache_dir keeps generated code across restarts. See the compiled tier guide.

How it scales, and what it costs

A floor is paid per concurrent runner, so the question a host actually has is whether it still pays with many of them. It does, and by the same factor throughout. QuickJS, 25 requests per worker, on 14 cores about 70% idle:

CPython, 20 requests per worker, at a floor of 1,000,000 and about 60% idle:

Two things to take from both tables. The floor is worth a constant factor at every worker count, 2.4x on QuickJS and about 3x on CPython, so it does not wash out under concurrency. And the scaling is sublinear in both arms alike: 7.1x without the floor against 6.8x with it on QuickJS, 4.4x against 4.3x on CPython. What a floor moves is the height of the curve, not its shape.

Do not read the ceilings as the runtime's. These runs had roughly 10 and 8 of 14 cores actually free, which is most of why the curves flatten where they do. The floor comparison survives that because both arms met the same machine in the same minute; an absolute scaling limit would not.

It costs less memory, not more, which is the opposite of what a per-runner ballast suggests. Fourteen workers peak at 129 MB with the floor against 204 without. That comparison is unfair to the floor twice over -- the floored arm finishes in 1164 ms against 2775, so it is sampled less often, and the ballast it adds is 34 MB that the unfloored arm never pays. Rerun with the request counts chosen to make the two arms the same length, 1156 ms against 1130, it is 122 MB against 195 to 210. The garbage a floor stops accumulating is simply larger than the heap it reserves.

Run your own with the throughput mode, and read the caveat in bench/paths/README.md first: a scaling curve cannot be made self-controlling by interleaving the way a latency sweep can, so it needs a quiet machine and there is no trick that substitutes for one.

What this project has not measured

Said plainly rather than filled with general advice, because a claim here cites test/audit/PERF.md and there is nothing to cite: allocator flags (+M*), scheduler binding (+sbt), scheduler counts and dirty schedulers have no measurement in this tree. If you measure any of them, that file is where the numbers go.