wasm_heap (wasm v0.3.0)

View Source

The garbage-collected object store.

You will not call this directly; wasm:pin/2 and wasm:release/2 are the parts you use. Read it when you want to know what a struct or an array costs, or why holding a reference needs a pin.

Why it is not a term inside the instance state

It was, and that was the most expensive thing in this runtime.

wasm_instance:set_mut/2 writes the #mut{} into ETS, ETS copies on insert, and the object store sat inside it. So one struct.set copied every object in the heap: 20.7 us at a thousand objects, 199 us at ten thousand, 1993 us at a hundred thousand. That is 19.7 ns per object in the heap, per mutating call, against 0.15 us for a call that only reads.

The store now lives here, mutated in place, and the handle is immutable: two ETS table ids and an atomics reference. It travels in #inst{}, the half of an instance that never changes, so an allocation or a field write leaves the state term untouched and the write-back never fires.

Why ETS, when array is faster per operation

array wins on both: 13.4 ns against 33.9 ns for a read, 50.6 ns against 93.0 ns for a write. It loses anyway, because those are the wrong numbers to compare. An array has to be written back somewhere to be shared, and writing it back copies it, so the cost that matters is per call and proportional to the heap. ETS rows are mutated in place. The read pays 20 ns more and the call stops paying milliseconds.

Layout

Up to two tables, because they are traversed differently:

  • objects, a set. A struct is {Id, s, TypeIdx, F0, F1, ...}, one tuple element per field, so reading a field copies the field and not the object. An array is its header only: {Id, a, TypeIdx, Len, Default}.
  • elements, an ordered_set keyed {Id, Index}, holding array elements that have actually been written. An array of a million defaults costs one row, and the ordering is what lets the collector walk one array's elements without scanning the table. A module declaring no array type never gets this table, because a table costs about a microsecond and instantiation is under three.

Identity

Ids come from an atomics counter and are never reused, so ref.eq stays an integer comparison. The sweep walks the table's rows rather than the id space, so it costs what is in the store and not what has ever been in it. What growing ids do still cost is the mark bitmap, which is sized to the highest id handed out. Reuse needs a free list, and it is not safe until every reference that leaves the runtime is pinned: today an unpinned one traps, and with reused ids it would silently name a different live object instead.

Summary

Types

An opaque handle. Immutable: the tables it names are what change.

Functions

Add Token as a holder of a heap this instance did not create.

Copy a range of elements from one array to another.

Fill a range with one value.

Write an element whose index is already known to be in range.

Bring this heap's charge in line with what it actually occupies.

Collect, minor unless the old generation has grown enough to warrant a major.

Drop an instance, releasing the heap once none is left.

Every instance registered with this heap.

Take a read lease on the store, answering whether one was taken.

Whether the next collection will trace the whole store.

Create a heap.

Create a heap charged to Token, whose Owner dying gives its pages back.

Allocate an array.

Keep an object alive across collections until it is released.

Every pinned reference, as collection roots.

How many read leases are outstanding. For tests and diagnostics.

Record that an instance's roots have to be traced when this heap collects.

Release the exclusive hold, leaving any read lease the caller had.

Record that a collection is wanted but could not be performed now.

Whether enough has been allocated since the last collection to be worth tracing. A call that allocates nothing answers in two atomics reads.

How many objects the store holds. For tests and diagnostics.

Take the store exclusively, from a caller already holding one read lease.

The kind and declared type of an object, for ref.test and ref.cast.

Give up a read lease.

Release one pin, letting the object be collected once nothing else holds it.

Release every pin, for when you scope references to a request.

Types

heap()

-nominal heap() :: {wasm_heap, ets:tid(), undefined | ets:tid(), atomics:atomics_ref()}.

An opaque handle. Immutable: the tables it names are what change.

Functions

acquire/3

-spec acquire(undefined | heap(), wasm_keeper:token(), pid() | none) ->
                 ok | {error, gone | instance_limit | keeper_unavailable}.

Add Token as a holder of a heap this instance did not create.

Linked instances share one store, so a heap has holders exactly as a shared memory does, and the keeper's rule that the strictest holder bounds it then applies without anything here knowing about ceilings.

allocs/1

-spec allocs(undefined | heap()) -> non_neg_integer().

array_copy/6

-spec array_copy(heap(), term(), non_neg_integer(), term(), non_neg_integer(), non_neg_integer()) -> ok.

Copy a range of elements from one array to another.

Both ranges are already checked, as they are for array_fill/5, so this does not ask again. wasm_exec used to hold the loop, and it read each element through array_get/3, which calls array_len/2, which is a table lookup per element re-answering the question the range check had just answered.

The source's default is read once rather than per element. Most elements of an array have no row of their own, so array_get/3 reaches the object table for every one of them, and a copy out of a defaulted array is that lookup and nothing else. The cost is that a whole-array fill of the source running concurrently, the only thing that changes a default, is not observed part way through a copy. Two processes writing one array while a third copies it is already unordered, and the snapshot below has the same property.

An overlapping copy behaves as though an intermediate copy were taken, which is what the specification asks for, and it does not take one: the direction decides it. Reading the whole source into a list first was the other way to get that answer, and it costs a list per copy.

array_default/2

-spec array_default(heap(), term()) -> term().

array_fill/5

-spec array_fill(heap(), term(), non_neg_integer(), non_neg_integer(), term()) -> ok.

Fill a range with one value.

A fill covering the whole array is not a loop at all: every element becomes the same value, which is precisely what the default means here, so it is one row update plus dropping whatever overrides existed. Arrays.fill and zeroing a freshly allocated array are both this shape.

The caller has already checked the range.

array_get/3

-spec array_get(undefined | heap(), term(), non_neg_integer()) -> term().

array_len/2

-spec array_len(undefined | heap(), term()) -> non_neg_integer().

array_set/4

-spec array_set(undefined | heap(), term(), non_neg_integer(), term()) -> ok.

array_set_unchecked/4

-spec array_set_unchecked(heap(), term(), non_neg_integer(), term()) -> ok.

Write an element whose index is already known to be in range.

Every bulk operation checks its whole range up front, because one that traps must leave the array untouched. Checking again per element then doubles the table operations to answer a question already answered.

charge(H)

-spec charge(undefined | heap()) -> ok | {error, term()}.

Bring this heap's charge in line with what it actually occupies.

Measured rather than accumulated, so a miscount cannot survive one call: what the tables report is the truth and the keeper is told to match it. That is the same shape wasm_keeper:init/1 uses to repair the page counter after a registry loss, and for the same reason.

Answers {error, Why} when the node budget or a holder's ceiling refuses the new size, and raises nothing. Whether a refusal is a trap depends on who is asking: a guest allocating gets one, and a collection must not, because it runs from the after of an invocation where an exception escapes wasm_error:capture/1 and leaves the runtime as a raw {wasm_error, _} rather than a value. refuse_if_over/1 is the mutation path's half of that.

collect/2

-spec collect(undefined | heap(), [term()]) -> ok.

Collect, minor unless the old generation has grown enough to warrant a major.

Minor. Objects allocated since the last collection are exactly the id range [watermark, next_id), because ids are handed out by a counter and never reused. So the nursery needs no bookkeeping at all: it is an integer range, and sweeping it is a loop over that range rather than a walk of the store.

A minor collection never traces an old object, which is what makes the pause proportional to what was just allocated rather than to everything alive. That is sound because an old object can only point at a young one through a write made since the last collection, and set_field/4 and array_set/4 record those.

Major. Traces and sweeps everything, and is where the long pause lives. It runs when the store has grown past gc_major_ratio times what it was after the last major (default 2), so a program with a stable live set almost never has one.

That is measured in rows and in bytes, because a store can be enormous and hold five of them. size/1 is a row count, and a workload replacing one large array per call never reached the row floor, never got a major collection, and so never had anything reclaimed: a minor one leaves the old generation alone by design. Four rounds of a fifty thousand element array left all four in the store, sixteen megabytes, with one reachable. The page count charge/1 already computes serves as the second unit, with its own floor, gc_min_major_pages.

Mark and sweep rather than copying, because object ids are handed out and compared by ref.eq, so they must not move. The mark set is an atomics bitmap and the worklist holds ids rather than values: the first collector kept its live set in a map and built its worklist with ++, costing 76 ns per object for the map alone and growing the calling process's BEAM heap by 8.5 words per object.

delete/2

-spec delete(undefined | heap(), term()) -> ok.

Drop an instance, releasing the heap once none is left.

Idempotent: wasm:destroy/1 is documented as safe to call twice.

get_field/3

-spec get_field(undefined | heap(), term(), non_neg_integer()) -> term().

instances/1

-spec instances(undefined | heap()) -> [term()].

Every instance registered with this heap.

is_heap/1

-spec is_heap(term()) -> boolean().

lease/1

-spec lease(undefined | heap()) -> boolean().

Take a read lease on the store, answering whether one was taken.

Answers false for a module with no object store at all, which is every module declaring no struct and no array type, so those pay one comparison.

major_due/1

-spec major_due(undefined | heap()) -> boolean().

Whether the next collection will trace the whole store.

new()

-spec new() -> heap().

Create a heap.

The tables are owned by the calling process, which is the process instantiating the module, so a heap has exactly the lifetime #inst.store already has and adds no new ownership rule.

new(Token, Owner)

-spec new(wasm_keeper:token(), pid() | none) -> heap().

Create a heap charged to Token, whose Owner dying gives its pages back.

A heap is a registry resource like a memory, for the reason wasm_engine's moduledoc gives: its objects are ETS rows, ETS is not process heap, and so nothing the BEAM offers can see them. Counting them somewhere private would drift the moment the creating process died without destroying, because the tables die with it. The keeper already owns the monitor, the holder set and the reconciliation that stop exactly that.

new_array/5

-spec new_array(heap(), non_neg_integer(), non_neg_integer(), term(), boolean()) ->
                   {objref, non_neg_integer()}.

Allocate an array.

Traced says whether its elements can hold references. An array of i32 or i8 cannot, so the collector never walks it: no scan of its elements, however many have been written. That is most of what a language like Java or Kotlin allocates, and walking a million-element byte array to find no references in it is the kind of work worth not doing.

new_struct/3

-spec new_struct(heap(), non_neg_integer(), [term()]) -> {objref, non_neg_integer()}.

pin/2

-spec pin(undefined | heap(), term()) -> ok.

Keep an object alive across collections until it is released.

A reference you are handed is not reachable from any root the runtime can see, so without this the next collection frees it and leaves you holding an id that names nothing. Call results, wasm:get_global/2 results and the values of an uncaught exception are pinned on the way out.

Reference counted, so a reference handed out twice needs releasing twice. The count lives beside the object rather than in the instance state, which is what stopped pinning from costing a state write-back on every call: it used to be a list rebuilt with lists:usort/1 per call that never shrank.

pins/1

-spec pins(undefined | heap()) -> [term()].

Every pinned reference, as collection roots.

readers/1

-spec readers(undefined | heap()) -> non_neg_integer().

How many read leases are outstanding. For tests and diagnostics.

register/3

-spec register(undefined | heap(), term(), term()) -> ok.

Record that an instance's roots have to be traced when this heap collects.

A heap outlives any one of its instances, because linked instances share it. Deleting it when the last one goes is what keeps that from leaking, and it is why delete/2 takes the instance that is leaving rather than just the heap.

What is stored is a summary, not the whole #inst{}. ETS copies on insert, and an instance record carries its type table, its compiled functions and its exports; registering one cost 1.9 us of a 5.7 us instantiation. The summary is the four fields a root scan actually reads.

release_exclusive/1

-spec release_exclusive(undefined | heap()) -> ok.

Release the exclusive hold, leaving any read lease the caller had.

request_collect/1

-spec request_collect(undefined | heap()) -> ok.

Record that a collection is wanted but could not be performed now.

set_field/4

-spec set_field(undefined | heap(), term(), non_neg_integer(), term()) -> ok.

should_collect/1

-spec should_collect(undefined | heap()) -> boolean().

Whether enough has been allocated since the last collection to be worth tracing. A call that allocates nothing answers in two atomics reads.

size/1

-spec size(undefined | heap()) -> non_neg_integer().

How many objects the store holds. For tests and diagnostics.

try_exclusive/1

-spec try_exclusive(undefined | heap()) -> boolean().

Take the store exclusively, from a caller already holding one read lease.

Succeeds only if that lease is the only one, which is what makes this an upgrade that cannot deadlock: there is nobody to wait for.

type_of/2

-spec type_of(undefined | heap(), term()) -> {struct | array, non_neg_integer()}.

The kind and declared type of an object, for ref.test and ref.cast.

A reference naming a slot that is not there is a trap, not a crash. It means a reference outlived the object it named, and reporting that as a case_clause three layers down says nothing about what went wrong.

unlease/2

-spec unlease(undefined | heap(), boolean()) -> ok | collect_now.

Give up a read lease.

Answers collect_now when this was the last reader out and somebody asked for a collection they could not perform themselves. The caller does it, because gathering roots is not this module's business.

unpin/2

-spec unpin(undefined | heap(), term()) -> ok.

Release one pin, letting the object be collected once nothing else holds it.

Releasing a reference that was never pinned is not an error, so you can release everything you have seen without remembering which ones counted.

unpin_all/1

-spec unpin_all(undefined | heap()) -> ok.

Release every pin, for when you scope references to a request.