atomvm-cbor 0.2.0 validation report

This report records the matched host and attached-device evidence collected on 2026-07-31 for the public 0.2.0 release candidate.

Provenance and methodology

• Baseline: public tag v0.1.1, commit 9a949de5fc6044c8ca1e4ca582b5584dffeefde5.
• Candidate: the committed PR #5 runtime tree selected by the exact-head gate.
• Candidate runtime source SHA-256: avm_cbor.erl is 25688378995023ec21412ee553a96b84b300aa8e772fb55e5b4027c7bd0bca85; avm_cbor_cont.erl is ba308f4beeb4963eda32aec908b89b2f8878d6747ff5b08766949c03fdef2b67.
• Host: Debian 13, x86_64, Intel Pentium Silver J5005, OTP 27.
• Attached runtime: AtomVM 0.6.6 with the same benchmark payload and harness for both versions.
• WaveShare baseline pack: v0.1.1 commit 9a949de5fc6044c8ca1e4ca582b5584dffeefde5, pack SHA-256 220eb28b8852abf677eefa7d623592a4d0a7958807d90d2f2a17dbce6db5fab1, serial capture SHA-256 545508c7ac50213e4a3e4e82e805a063714edb595b0b2a326944b759a2b2816d.
• Each table value is the median of five complete paired runs. Host run values are themselves medians or p95 values from 31 samples after 1,000 warmups. Existing ESP32-S3 and RP2040 values are per-call means after one warmup; WaveShare baseline and candidate values are per-call means from matched benchmark-only/final captures.
• Run order alternated between baseline and candidate to reduce order bias.

Raw logs, CSV files, pack hashes, flash logs, and serial captures are retained in the operator evidence directory on the Dell hardware host.

Host results

Lower timings are better. Changes compare the candidate with v0.1.1.

The exact-head CI benchmark independently enforces the unchanged +5% regression limit under OTP 29. It derives the baseline from reachable SemVer history rather than a fixed version name: read VERSION, normalize optional v, exclude the current version, and choose the unique greatest lower SemVer. The gate fails closed when no baseline exists or equal-precedence candidates make the result ambiguous. Logs and generated filenames identify both versions and commit SHAs.

The release workflow always runs hygiene and routing policy first. Relevant runtime or benchmark changes then run performance alone; a passing result unlocks the other selected OTP, coverage, AtomVM, ESP-IDF, and package jobs, which may use both self-hosted runners in parallel. Documentation-only updates select package validation without replaying unrelated runtime builds. Tag and manual release gates still select the full chain, and the +5% threshold is unchanged.

The paired host gate starts every baseline/current capture in an identical single-scheduler Erlang VM and alternates their order across five runs. This stabilizes p95 sampling without changing the threshold. Feature availability is probed from the selected baseline tag instead of being inferred from a fixed release name.

Attached-device results

Hardware identities:

• ESP32-S3 QFN56 rev 0.2, 160 MHz, 8 MB PSRAM, CH340 adapter.
• WaveShare ESP32-S3-DEV-KIT-N32R16V, 160 MHz, 32 MB OPI/DTR flash, 16 MB octal PSRAM, AtomVM 0.6.6 with ESP-IDF v5.4.3.
• RP2040 B2, 16 MB flash, flash ID E460A8534F5C2139.

The WaveShare values are per-call averages from matched v0.1.1 and exact final-code captures; ESP32-S3 and RP2040 values are medians of five paired runs. The WaveShare captures completed without a watchdog, panic, abort, backtrace, or out-of-memory marker. Its final decode/1 value is 21.41% faster than its v0.1.1 baseline; it is also 1.02% above the pre-remediation measurement of 4553.74 us/op, inside the unchanged 5% regression allowance.

Interpretation of the older ESP32-S3 decode regression

The older ESP32-S3 result is reproducible across the five paired captures: v0.1.1 measured 8508.88-8509.32 us/op and the candidate measured 8683.58-8687.12 us/op. The 174.70 us/op median difference is therefore reported as a real 2.05% target-specific regression rather than dismissed as noise.

This 20-byte benchmark payload is a mixed map with text keys, text values, and a multi-byte integer. It cannot use the fixed-cost preferred-one-byte-unsigned container path added for the physical watchdog boundary. It instead exercises the general decoder, including 0.2.0's fail-closed global node and cumulative string-byte charging, declared-child admission, and normalized policy state. Those security/correctness checks are additional work, but they are not proven to be the only cause: the same code and payload improve on RP2040 and WaveShare, and the corresponding host decode workloads improve as well.

Cross-board absolute values are not a controlled A/B experiment. The older 160 MHz ESP32-S3 uses AtomVM 0.6.6, ESP-IDF 5.4.1, 80 MHz DIO flash, and 8 MB PSRAM. The 160 MHz WaveShare uses AtomVM 0.6.6, ESP-IDF 5.4.3, OPI/DTR flash, and 16 MB octal PSRAM. Flash, external-memory, firmware, and cache behavior can all affect AtomVM interpreter throughput. A single hardware cause for the 8683.68 versus 4600.30 us/op difference would require a controlled same-board firmware/configuration experiment and is not claimed here.

The release accepts the older-board result under the unchanged, pre-declared 5% regression allowance because removing the bounded accounting would weaken the security contract. This does not claim that the regression is desirable or that later board-specific profiling could not recover it.

Full attached validation

The exact candidate source also completed the AtomVM target suite, benchmark, and 40-round retained-growth soak on both boards. Neither serial capture contained a watchdog, backtrace, panic, abort, or out-of-memory marker.

These are bounded measurements on the named devices, not universal latency or leak-free guarantees.

Maximum-bound result

The WaveShare board passed four rounds of depth 128, exactly 4096 charged nodes in flat-array and tagged-map forms, 64 KiB byte/text strings, and an exactly 1 MiB admitted input through both synchronous and continuation APIs. Standard array decode stayed between 17122 and 17199 us; the tagged map stayed between 3025944 and 3027233 us. The full run then completed the release benchmark and 40-round soak with a flat retained-growth gate.

The same fixed-cost path removed the former watchdog boundary on the 8 MB-PSRAM ESP32-S3 (16095 us array; 2340921 us map). That target then failed while the harness allocated the 1 MiB input binary. RP2040 likewise cannot materialize a 1 MiB Erlang binary in 264 KiB SRAM. These are transport/runtime capacity limits, not continuation failures: constrained deployments must reject a smaller fixed byte ceiling before buffering the request. The release therefore does not make a universal 1 MiB zero-OOM claim.