wasm_core (wasm v0.3.0)
View SourceThe Core Erlang back end: it generates the forms for a set of WebAssembly functions, and compiles them.
wasm_jit decides when to compile and wasm_code_slots decides which
module name the result gets; this decides what the code says. module/4 and
its longer arities take the functions and answer a loaded module;
can_compile/2 and supported/1 are the filter that says whether a function
is inside the subset at all, and wasm_jit asks them before building anything.
Where things are
| you want | look at |
|---|---|
| whether a function can be compiled | can_compile/2, supported/1 |
| the generated shape of one instruction | binop/4, cmp/1, access/7, call_op/2 |
| a whole function's forms | function/4, forms/5 |
| the module around them, and the compiler run | module/4, run_compiler/4 |
| the atoms a name can be | fun_name/1, frame_name/1, atoms/0 |
| the bounds a generator refuses past | limits/0, and the table below |
The rest of this doc is about naming, which is one concern of several here but the one that can leak the node.
Names, and why they are bounded
Read this before you generate a name, because a Core function identifier has to be an atom and the atom table is node-wide and never reclaimed: a name derived from a module's own bytes is a permanent leak with a remote attacker holding the tap.
wasm_code_slots solves that for module names by writing sixteen of them out
longhand. Function and frame names cannot be written out longhand, because
QuickJS has 1666 compilable functions, so they are computed instead -- once, at
first use, from a bound that is a literal in this file. The property is the same
one and is what matters: the set of atoms this module can ever create is
decidable by reading it, and nothing user-controlled reaches a name.
The bounds, and where they came from
bench/paths/subset.erl reports the shapes a generator is bounded by, over the
Rust plugin and over QuickJS:
| plugin max | qjs max | qjs p99 | |
|---|---|---|---|
| bound on continuation arity | 20 | 71 | 37 |
| control frames per function | 118 | 1016 | 165 |
| control nesting | 18 | 257 | 30 |
| compilable functions | 150 | 1666 |
Arity is not the constraint it looks like. The worst function in QuickJS
would generate a 71-argument continuation against the BEAM's limit of 255, so
?MAX_ARITY costs no coverage at all and exists to refuse the pathological
rather than to shape the common case. The decoder admits a million locals, which
wasm_decode expands, and that is what a bound has to survive rather than what
it has to accommodate.
Frame names are bounded by nesting, not by count. A function with 1016
control frames needs far fewer than 1016 names, because Core letrec scoping
puts a sibling frame's name out of scope: names have to be unique only along a
nesting path, which is 257 deep at worst.
A module or function past a bound is not compiled. That is the same answer as every other refusal here: interpret it.
Summary
Functions
Every atom this module can create. The whole set, for tests to assert on.
Whether a function can be compiled, and what it would cost if so.
The Core Erlang this unit lowers to, before the OTP compiler sees it.
As forms/5, naming the generated module that holds the rest of this one.
As forms/6, naming the head of the chain as well as the next link.
As forms/7, saying which other unit holds each function this one does not.
The name of the Nth control frame along one nesting path within a function.
The name of the Nth function of a compiled unit, counting from zero.
Every bound, so a caller can refuse before it starts rather than part way.
Generate and compile one BEAM module for a set of WebAssembly functions.
As module/4, choosing how hard the OTP compiler works.
As module/6, naming the generated module that holds the rest of this one.
As module/7, naming the head of the chain for a crossing to re-enter at.
As module/8, saying which other unit holds each function this one does not.
As module/9, bounding the heap of the process that runs the OTP compiler.
Every operation wasm_exec:op1/2 and op2/3 are expected to implement, with
the number of operands it takes.
Kill Child if Owner dies first, and stop as soon as either does.
Whether the generator can emit code for one lowered instruction.
Functions
-spec atoms() -> [atom()].
Every atom this module can create. The whole set, for tests to assert on.
-spec can_compile(#fn{nparams :: non_neg_integer(), nresults :: non_neg_integer(), defaults :: [term()], body :: [tuple() | atom()] | {lazy, [term()]}, raw :: [term()], idx :: non_neg_integer(), type :: term(), frame :: term()}, [term()]) -> {ok, map()} | {unsupported, term()} | {limit, atom()}.
Whether a function can be compiled, and what it would cost if so.
Answers a diagnosis rather than a boolean, because a refusal that is expected
and a refusal that is a defect look identical as false, and the conformance
suite's force-eligible mode has to tell them apart:
{ok, Metadata}— nothing known to refuse. Generation may still answer{limit, _}, because a continuation's arity is only known once the compile-time operand stack has been walked.{unsupported, Instr}— outside the subset. Expected, and the common answer.{limit, Reason}— inside the subset but past a bound. Expected, and rare: QuickJS's worst function nests 257 deep against a bound of 512.
-spec forms(module(), [term()], map(), map(), binary() | non_neg_integer()) -> {ok, cerl:c_module()} | {error, term()}.
The Core Erlang this unit lowers to, before the OTP compiler sees it.
Use it when you have changed a clause of the generator and want to know what it
produced. module/6 calls this and then compiles the result, so what you print
here is what runs, and wasm_jit:dump/1 is the way to reach it from an
instance you already have:
{ok, Core} = wasm_core:forms(wasm_code_0, Unit, Sigs, TSigs, 0),
io:format("~s~n", [core_pp:format(Core)]).This exists because the alternative is reading 1,284 lines of cerl calls and
imagining the tree. A differential test tells you the generator is wrong; this
tells you how.
-spec forms(module(), [term()], map(), map(), binary() | non_neg_integer(), undefined | module()) -> {ok, cerl:c_module()} | {error, term()}.
As forms/5, naming the generated module that holds the rest of this one.
undefined for a module compiled as a single unit, which is every one until
something asks for shards.
-spec forms(module(), [term()], map(), map(), binary() | non_neg_integer(), undefined | module(), module()) -> {ok, cerl:c_module()} | {error, term()}.
As forms/6, naming the head of the chain as well as the next link.
Head is where a crossing back into the interpreter says to re-enter, and it is
the first unit rather than this one. The chain runs one way, so a caller that
starts in the middle can only reach what is below it: entering at the head is
what makes every function reachable from every shard.
-spec forms(module(), [term()], map(), map(), binary() | non_neg_integer(), undefined | module(), module(), #{non_neg_integer() => module()}) -> {ok, cerl:c_module()} | {error, term()}.
As forms/7, saying which other unit holds each function this one does not.
A call to one of those is emitted as a direct call into that unit rather than a crossing back into the interpreter, which is the whole difference between splitting being worth it and not.
-spec frame_name(non_neg_integer()) -> atom().
The name of the Nth control frame along one nesting path within a function.
-spec fun_name(non_neg_integer()) -> atom().
The name of the Nth function of a compiled unit, counting from zero.
Named by position in the unit rather than by WebAssembly index, so a module that compiles a hundred of its five thousand functions uses a hundred names.
-spec limits() -> #{atom() => pos_integer()}.
Every bound, so a caller can refuse before it starts rather than part way.
-spec module(module(), [{non_neg_integer(), non_neg_integer(), #fn{nparams :: non_neg_integer(), nresults :: non_neg_integer(), defaults :: [term()], body :: [tuple() | atom()] | {lazy, [term()]}, raw :: [term()], idx :: non_neg_integer(), type :: term(), frame :: term()}, [term()]}], #{non_neg_integer() => {non_neg_integer(), non_neg_integer()}}, #{non_neg_integer() => {non_neg_integer(), non_neg_integer()}}) -> {ok, binary()} | {error, term()}.
Generate and compile one BEAM module for a set of WebAssembly functions.
Unit is [{Pos, Idx, #fn{}, IR}], where Pos numbers the functions within
this unit and Idx is the WebAssembly function index the generated invoke/5
dispatches on. Names come from the pools, by Pos, so a module compiling a
hundred of five thousand functions uses a hundred names.
-spec module(module(), [term()], map(), map(), baseline | full, binary() | non_neg_integer()) -> {ok, binary()} | {error, term()}.
As module/4, choosing how hard the OTP compiler works.
baseline skips the SSA optimiser. On QuickJS's 1666 functions that was
119.9 seconds down to 45.9 for 10% slower code, which looked like an easy
trade and is not: the same option costs 86% on bench/cross/loop.wasm,
2.51 nanoseconds an iteration against 4.66. Ten percent was what a bytecode
interpreter's dispatch loop happens to lose; tight arithmetic is what the SSA
optimiser is actually for.
So full is the default. It is affordable because the two changes that came
after this one made it so: compilation is off the calling process, and it is
sized to the functions that ran rather than the functions that exist, so the 74
seconds is about 8 and nobody waits for it. baseline remains for a caller who
would rather have the module sooner.
-spec module(module(), [term()], map(), map(), baseline | full, binary() | non_neg_integer(), undefined | module()) -> {ok, binary()} | {error, term()}.
As module/6, naming the generated module that holds the rest of this one.
A wasm module compiled into several units chains them: each answers for the
functions it holds and hands anything else to Next, which is a literal it was
built with. The last one answers not_compiled and the caller interprets.
-spec module(module(), [term()], map(), map(), baseline | full, binary() | non_neg_integer(), undefined | module(), module()) -> {ok, binary()} | {error, term()}.
As module/7, naming the head of the chain for a crossing to re-enter at.
-spec module(module(), [term()], map(), map(), baseline | full, binary() | non_neg_integer(), undefined | module(), module(), #{non_neg_integer() => module()}) -> {ok, binary()} | {error, term()}.
As module/8, saying which other unit holds each function this one does not.
-spec module(module(), [term()], map(), map(), baseline | full, binary() | non_neg_integer(), undefined | module(), module(), #{non_neg_integer() => module()}, #{max_heap_words => pos_integer()}) -> {ok, binary()} | {error, term()}.
As module/9, bounding the heap of the process that runs the OTP compiler.
Opts takes max_heap_words, and a unit whose compiler exceeds it answers
{error, {limit, {compile_memory, Words}}} rather than dying. Absent means no
ceiling, which is the default and what every caller but wasm_jit passes.
-spec ops() -> [{atom(), 1 | 2}].
Every operation wasm_exec:op1/2 and op2/3 are expected to implement, with
the number of operands it takes.
The arity is answered here rather than guessed by the caller, because guessing it is how a unary operation gets checked as a binary one and silently passes.
Kill Child if Owner dies first, and stop as soon as either does.
Exported for wasm_jit's pmap, which has the same problem one rung up: its
shard workers are spawned monitored and unlinked, so a killed coordinator would
leave them compiling toward slots nobody owns.
Whether the generator can emit code for one lowered instruction.
Control instructions carry a body and are answered true here; whether their
contents are supported is can_compile/2's recursion, not this.