atomvm-cbor 0.3.0 validation report

This report records the host release gate, the controlled same-device 0.2.0 to 0.3.0 comparison collected on 2026-08-23, and the retained functional and bounded-soak evidence for the public 0.3.0 release.

Release identity

Baseline: public annotated tag 0.2.0, commit 50cae38fc0d846e41fe61c9d5d80229c6cff91f2.
Candidate: public annotated tag 0.3.0, commit 06333306be6d0f83d9b0b257afdc29f82bcd22ea.
Exact-tag release gate: run 32647464389, performance job 97214051323.
Performance artifact identity: 0.2.0-50cae38fc0d8--0.3.0-06333306be6d.

Provenance and methodology

Host

Dell hardware runner, Intel Pentium Silver J5005, Linux x86_64.
Erlang/OTP 29, one normal scheduler, one dirty CPU scheduler, and one dirty I/O scheduler in the workflow's digest-pinned Erlang container.
Five alternating fresh-VM baseline/candidate pairs. Each workload uses 31 samples after 1,000 warmups; the table reports the median of the five run-level medians and the median of the five run-level p95 values.
Lower latency is better. The fail-closed limit remains 5% for both median and p95 on workloads present in both releases.

Attached devices

Both releases were built from the exact public tags above.
AtomVM 0.6.6 source commit: ff993a80963298b532c1e573f883951ecaac9fef.
ESP32-S3 firmware used ESP-IDF v5.4.3. Both ESP32-S3 CPUs ran at 240 MHz; flash and PSRAM ran at 80 MHz.
RP2040 firmware used the AtomVM Pico SDK fork at 1d68df97567bcadb89eaf18d4caead0d45151cb8, with mbedTLS at a77287f8fa6b76f74984121fdafc8563147435c8 and TinyUSB at 86c416d4c0fb38432460b3e11b08b9de76941bf5. Its system clock was explicitly configured to 133 MHz with a 1596 MHz PLL VCO and 6/2 post dividers.
These are the manufacturers' supported upper CPU clock limits, not overclocks: the ESP32-S3 datasheet specifies up to 240 MHz and the RP2040 specification specifies up to 133 MHz.
The same benchmark source was used by both releases. Harness SHA-256: 9ff77b1a96b155ebf6006b9929b2054c3adf3a7b032a913a8f1b39d8bc0e82e6.
Pack SHA-256 identities:
- 0.2.0: 017c7c63f812e470b863f8459b2b4c484eee892baf7fd4a19d60aee5d90a659a
- 0.3.0: 1dc94835d9023d721b1c741d2ac78a2c866da7eda42e43bd5ae835f3173f10f1
Each device completed five paired captures. Order alternated baseline/candidate, candidate/baseline, baseline/candidate, candidate/baseline, baseline/candidate to reduce order and thermal bias.
Common and partial workloads use 50 timed iterations after warmup; descriptor accessors use 250 timed iterations after warmup. Values are microseconds per call.
All 30 captures contain one begin marker, 14 benchmark rows, and one end marker. None contains a watchdog, panic, abort, backtrace, out-of-memory, or OOM marker.

Raw serial captures, build logs, toolchain revisions, pack files, firmware, and hashes are retained in the operator evidence directory on the Dell hardware host. The exact clock inputs are committed under hardware/benchmark.

Host results

Workload	0.2.0 median ns	0.3.0 median ns	Change	0.2.0 p95 ns	0.3.0 p95 ns	Change	Gate
`scalar_decode`	357	352	1.40% faster	363	357	1.65% faster	PASS
`scalar_encode`	159	156	1.89% faster	164	162	1.22% faster	PASS
`string_decode_64`	1089	1120	2.85% slower	1107	1139	2.89% slower	PASS
`string_encode_64`	800	790	1.25% faster	816	803	1.59% faster	PASS
`nested_decode`	2705	2751	1.70% slower	2737	2795	2.12% slower	PASS
`nested_encode`	6292	6262	0.48% faster	6339	6316	0.36% faster	PASS
`sequence_decode_32`	4270	4238	0.75% faster	4327	4310	0.39% faster	PASS
`partial_decode_nested`	3227	3272	1.39% slower	3271	3319	1.47% slower	PASS
`deterministic_map_encode`	9261	9350	0.96% slower	9331	9433	1.09% slower	PASS
`deterministic_map_decode`	5006	5047	0.82% slower	5073	5100	0.53% slower	PASS
`malformed_declared_array`	506	489	3.36% faster	515	500	2.91% faster	PASS

Overall host regression gate: PASS. Every directly comparable median and p95 remained within the unchanged 5% limit.

New 0.3.0 host workloads

These APIs do not exist in 0.2.0. Their baseline is therefore N/A; no percentage change is invented.

Workload	0.2.0	0.3.0 median ns	0.3.0 p95 ns
`encode_with_size_nested`	N/A	6359	6408
`encode_sequence_32`	N/A	6251	6328
`sequence_fold_32`	N/A	7966	8034
`validate_all_nested`	N/A	3186	3217
`partial_map_fold`	N/A	5105	5183
`partial_array_fold`	N/A	4189	4233
`partial_select`	N/A	6253	6309
`partial_map_find`	N/A	6545	6627
`partial_array_nth`	N/A	4139	4200

Attached-device results

Each row below compares the exact public tags on the same named physical device, firmware image, runtime, clock configuration, benchmark harness, and payload.

Board	Exact physical identity	CPU	Flash / PSRAM
ESP32-S3 N16R8	QFN56 rev 0.2, MAC `1c:db:d4:5b:f5:d0`, native USB identifier `1C:DB:D4:5B:F5:D0`	240 MHz	16 MiB DIO flash / 8 MiB octal PSRAM, both 80 MHz
WaveShare N32R16V	ESP32-S3-DEV-KIT-N32R16V, MAC `90:e5:b1:d8:48:b0`, CH340 UART serial `5B61092782`	240 MHz	32 MiB OPI/DTR flash / 16 MiB octal PSRAM, both 80 MHz
RP2040 E462…	RP2040 B2, BOOTSEL serial `E0C9125B0D9B`, flash ID `E46254C5C32D122C`	133 MHz	16 MiB flash / 264 KiB SRAM

Function	ESP32-S3 N16R8 0.2.0 µs	0.3.0 µs	Change	WaveShare N32R16V 0.2.0 µs	0.3.0 µs	Change	RP2040 E462… 0.2.0 µs	0.3.0 µs	Change
`encode/1`	8849.46	8844.02	0.06% faster	3507.30	3508.00	0.02% slower	4824.60	4791.26	0.69% faster
`decode/1`	10063.52	10044.02	0.19% faster	3931.22	3935.36	0.11% slower	6082.16	6067.22	0.25% faster
`partial_decode/1`	15575.38	15575.92	0.00% slower	6325.48	6327.62	0.03% slower	9261.06	9346.80	0.93% slower
`partial_decode/2`	15737.26	15738.90	0.01% slower	6373.28	6373.64	0.01% slower	9275.70	9372.16	1.04% slower
`partial_deep_decode/1`	12227.04	12219.28	0.06% faster	4972.54	4985.38	0.26% slower	7742.00	7786.84	0.58% slower
`partial_value_bytes/1`	1017.20	1011.64	0.55% faster	451.96	463.16	2.48% slower	557.50	555.89	0.29% faster
`partial_contents/1`	1260.14	1252.28	0.62% faster	511.97	522.50	2.06% slower	498.50	503.04	0.91% slower
`partial_skip/1`	509.06	480.23	5.66% faster	227.07	232.11	2.22% slower	226.12	221.10	2.22% faster
`partial_type/1`	510.14	479.82	5.94% faster	226.62	231.48	2.14% slower	226.05	221.50	2.01% faster
`partial_count/1`	508.55	480.30	5.56% faster	227.51	232.16	2.04% slower	228.13	221.88	2.74% faster
`partial_tag/1`	554.26	605.99	9.33% slower	270.87	309.82	14.38% slower	250.46	262.50	4.81% slower
`partial_size/1`	568.37	618.16	8.76% slower	285.07	323.83	13.60% slower	263.27	270.00	2.56% slower
`partial_offset/1`	508.95	479.53	5.78% faster	227.21	231.56	1.91% slower	225.49	223.25	0.99% faster
`partial_length/1`	511.07	477.45	6.58% faster	228.18	231.84	1.60% slower	228.41	224.42	1.75% faster

Comparable-workload totals:

Scope	Faster rows	Slower rows	Net average change
ESP32-S3 N16R8	10	4	-0.92%
WaveShare N32R16V	0	14	+3.06%
RP2040 E462…	8	6	-0.01%
All 42 board/workload results	18	24	+0.71%

The net average is the unweighted arithmetic mean of the signed per-row changes in this exact comparison matrix. Negative is faster; positive is slower. Current-only workloads with an N/A baseline are excluded.

New 0.3.0 attached-device workloads

These APIs do not exist in 0.2.0, so baseline and percentage remain N/A. Their absolute 0.3.0 results are measured separately on every target.

Workload	0.2.0 baseline	ESP32-S3 N16R8 0.3.0 median µs	WaveShare N32R16V 0.3.0 median µs	RP2040 E462… 0.3.0 median µs
`encode_with_size_nested`	N/A	8553.46	3487.34	4817.44
`encode_sequence_32`	N/A	22377.74	8938.22	10792.68
`sequence_fold_32`	N/A	51161.94	20206.16	27620.30
`validate_all_nested`	N/A	20292.08	7699.96	9819.90
`partial_map_fold`	N/A	33097.00	12651.24	17685.38
`partial_array_fold`	N/A	29531.66	11219.80	15642.32
`partial_select`	N/A	36947.22	14595.60	20275.32
`partial_map_find`	N/A	36417.72	14349.28	19772.56
`partial_array_nth`	N/A	27372.00	10498.66	14340.08

Evidence identities:

Candidate-only harness SHA-256: 299fec5a1782c62e6afd03ab134b8a9711c4dff465cfdf3e054095aaa1234d65.
Candidate-only pack SHA-256: a92c4b93d3ce03f64529c18be63b9b42f1034fc55add0d108df6b51a14b21e59.
Evidence bundle: candidate-api-three-targets-06333306be6d0f83d9b0b257afdc29f82bcd22ea.tar.gz (SHA-256 16b762a265d73ea53a9f9491722093222b9df87c0cc62877636acbd21c35e90e), Dell operator evidence: results/candidate-api/.
ESP32-S3 N16R8: 5 captures, log-set SHA-256 8ebad927d7bce1e592a50425205e80bf50355dec469b1d13ac49557b21ed6ab4; Dell operator evidence: results/candidate-api/esp32-s3-n16r8-candidate-api-run[1-5].log.
WaveShare N32R16V: 5 captures, log-set SHA-256 f7d4255126b375f76ff872806b77af8d2c97c9c90c124d0ce8f15a5ebdd799a4; Dell operator evidence: results/candidate-api/waveshare-n32r16v-candidate-api-run[1-5].log.
RP2040 E462…: 5 captures, log-set SHA-256 789e570e4622f75e27da6497d88fc434cfa6ec0415095fdacb71a133f317ce9d; Dell operator evidence: results/candidate-api/rp2040-candidate-api-run[1-5].log.

Manifest: docs/benchmarks/data/0.3.0.json (SHA-256 d231f731225b34b86fee48db3221341333f17533f662596929bf899b9786bfa6).

Interpretation

Core encode/1 and decode/1 latency is effectively flat on all three devices; every change is below 1%.
The partial decode paths remain within 1.04%.
partial_tag/1 and partial_size/1 are reproducible target-specific regressions and are reported rather than hidden. Across five runs, partial_tag/1 moved from 553.46-556.57 to 604.18-606.70 µs on ESP32-S3 N16R8, from 270.78-271.37 to 309.77-309.83 µs on WaveShare, and from 249.88-250.56 to 261.84-262.64 µs on RP2040. partial_size/1 shows the same stable direction: 561.07-569.02 to 617.77-619.63 µs, 284.64-285.34 to 323.63-323.95 µs, and 262.84-263.87 to 269.59-270.51 µs respectively.
These accessor measurements are sub-millisecond operations and are not part of the host release threshold. The exact public-tag host gate passed all comparable workloads under the unchanged 5% median/p95 limit.
Absolute timings must not be compared across boards: flash mode, external memory, cache behavior, and the AtomVM port all affect throughput.

Retained functional and bounded-soak evidence

The existing 14-row exact-tag comparison was not replayed to regenerate this report. The separate candidate-only captures above measured only the nine 0.3.0 APIs that the common harness did not cover.

The retained 0.3.0 release-candidate hardware validation is run 32586422115 at commit 12903636339edb1172167d3905907ff583ffd7d2. It completed the AtomVM target suite, benchmark, and 40-round retained-growth soak on the WaveShare N32R16V and RP2040 without watchdog, panic, abort, backtrace, or out-of-memory markers. Because that retained run did not cover the separate ESP32-S3 N16R8, the missing functional suite and 40-round soak were captured on that exact device from public tag commit 06333306be6d0f83d9b0b257afdc29f82bcd22ea.

Board	First/late heap maximum	First/late memory maximum	Rounds	Result
ESP32-S3 N16R8	63 / 63 words	492 / 492 bytes	40	pass
WaveShare N32R16V	63 / 63 words	492 / 492 bytes	40	pass
RP2040 E462…	63 / 63 words	484 / 484 bytes	40	pass

Evidence identities:

ESP32-S3 N16R8: Dell exact-tag supplemental functional/soak capture, 2026-08-23, source 06333306be6d0f83d9b0b257afdc29f82bcd22ea, esp32-s3-n16r8-functional-soak-06333306be6d0f83d9b0b257afdc29f82bcd22ea.tar.gz (SHA-256 7e138061ae7dcf7452bfd5e2410731d300516a0c85d0356f051fc48212449ac7).
WaveShare N32R16V: retained release-candidate run 32586422115, ESP job 97064781219, source 12903636339edb1172167d3905907ff583ffd7d2, esp32-s3-hardware-results-12903636339edb1172167d3905907ff583ffd7d2.tar.gz (SHA-256 51203bb68c0fd89f9b9ff909e6fa97b96a906b0db74dfb5a16311cab49ed1572).
RP2040 E462…: retained release-candidate run 32586422115, RP2040 job 97064781214, source 12903636339edb1172167d3905907ff583ffd7d2, rp2040-hardware-results-12903636339edb1172167d3905907ff583ffd7d2.tar.gz (SHA-256 0bd6594e9012bc967985fcf7fdd35f7ea4e5a470c928f29e3866896a98fcb238).

The retained WaveShare archive uses the workflow's generic esp32-s3 platform name; its discovery log records UART serial 5B61092782, binding it to the WaveShare N32R16V. These are bounded measurements on named devices, not universal latency or leak-free guarantees.

The historical 160 MHz evidence, maximum-bound measurements, and capacity limits remain unchanged in the 0.2.0 report. They are not relabelled as 240 MHz evidence.

Build and evidence identities

Item	SHA-256
ESP32-S3 N16R8 clock configuration	`95352d7732c317d2ba10e27ab50a7f041c705cd5804ea19ce70b98a74901d6c1`
WaveShare N32R16V clock configuration	`6ad001f7ceef09509656110088a4302af46f4eda5f1096411929b4b9a2942d43`
RP2040 133 MHz CMake configuration	`2bb06526cd054769b4b65fae746c5c802ce31ce0596c20d2952e1cc00c99ba10`
ESP32-S3 N16R8 bootloader	`321f9d9f435608c0fad3f4ebc55792375dda60bd9e8191f3916a070741efec5b`
ESP32-S3 N16R8 AtomVM image	`a034900e66ef6da9aa2fc18665f2cfa98eda60c84fb9e4214dd67864b73f9b0f`
WaveShare N32R16V bootloader	`e56a7e655af7866af85fd28c4d6ef32e57184a6814b587368b6ec729bbeed77e`
WaveShare N32R16V AtomVM image	`067fc155a8eb8691bb6a0086aa3f978089a37a82a7f48444a8f855498e9a553b`
Shared ESP32 partition table	`607f264e82a8d7cd81e27377548ad036081a5e483b3ac020062a7186b1191378`
RP2040 AtomVM UF2	`98cbf02970d80f47b9bc229fffa99c8a8520ab032cef5794a6b8916e88a66269`
RP2040 AtomVM binary	`8f4003e4bbf0af28bf8435e5ce37c3775e745222b3594f09d2c9dd5d21fa0c68`

Reproduction policy

The canonical host comparison remains:

BENCHMARK_RUNS=5 BENCHMARK_NOISE_TOLERANCE_PERCENT=5 \
  bash scripts/benchmark-remediation.sh

Release evidence documentation is generated from the fail-closed structured manifest at docs/benchmarks/data/0.3.0.json:

python3 scripts/release-evidence.py --write
python3 scripts/release-evidence.py

The first command regenerates the checked blocks in this report, the release notes, and the README. The second is check-only and is also called by scripts/check-release-metadata.sh. It rejects a missing target, workload, five-run candidate-only result, evidence hash, or 40-round functional/soak row.

For hardware, use the committed board-specific clock configuration, exact public tag packs, AtomVM/toolchain revisions, paired order, warmup, and iteration counts recorded above. Do not compare captures made on different physical devices or silently substitute historical clock settings. Existing valid release evidence is processed into the report; benchmarks are not repeated merely to regenerate documentation.

← Previous Page atomvm-cbor 0.2.0 validation report

Next Page → Benchmark Report: v0.1.1