Benchmarks

This page keeps public benchmark summaries. Raw benchmark logs and one-off profiling runs are intentionally not committed, so these are historical reference points rather than a self-contained reproducible artifact.

Use the commands and commit endpoints below for matched reruns; do not treat the recorded numbers as universal hardware claims. Throughput and latency depend on VM type, local NVMe availability, shard count, client concurrency, pipeline depth, payload size, and resource guards.

FerricStore PubSub: compact native pipeline

These loopback TCP measurements use pipeline depth 8 and 256-byte messages. Every subscriber acknowledgement, publish count, pushed delivery, and slow-consumer isolation check is validated by the runner.

The batch-aware comparison used 9dfa241f as its per-item endpoint and 5e189fa8 as its batch-aware endpoint. The negotiated batch-event comparison used f46aa319 and the runner's BENCH_PUBSUB_BATCH_EVENTS=0/1 switch. The recorded host was an Apple M4 Max. A representative matched command is:

BENCH_PUBSUB_CONCURRENCIES=1,4 \
BENCH_PUBSUB_PUBLISHES_PER_PUBLISHER=4096 \
BENCH_PUBSUB_LOAD_FANOUT=8 \
BENCH_PUBSUB_PUBLISH_PIPELINE=8 \
BENCH_PUBSUB_PIPELINE_ENCODING=compact \
BENCH_PUBSUB_SKIP_SLOW_PHASE=1 \
MIX_ENV=bench mix run --no-start bench/native_pubsub_load_bench.exs

Run fanout 1 and 8 separately and retain at least three samples per cell. For the depth-1,024 negotiated gate, set 65,536 publishes per publisher, pipeline depth 1,024, and toggle BENCH_PUBSUB_BATCH_EVENTS between 0 and 1.

Protocol-only comparison before batch-aware fanout

Batch-aware exact-channel fanout

The retained optimization resolves the exact subscriber list once per compact pipeline, reserves outbound capacity once per subscriber batch, enqueues one BEAM mailbox message per subscriber, and writes the batch's existing individual event frames in one socket send. It does not change the wire format. A depth-8, fanout-8 request therefore reduces 64 guarded reservations and mailbox messages to 8. Pattern subscriptions conservatively retain the original per-publish path so exact and pattern event ordering stays unchanged.

The following are three-sample medians from a matched compact-mode-35 comparison; the only benchmark toggle was ordinary per-item publish versus batch-aware publish:

Typed native pipelines containing only PUBLISH now reuse the same bounded batch fanout path. This preserves the typed wire format for older SDKs while removing the former per-item delivery overhead:

Sustained batch-size gate

Larger explicit publish_many requests reduce the remaining per-subscriber socket-write cost. With 65,536 publishes per publisher, 256-byte payloads, and fanout 8, the production-default activity log, and compact mode 35, the matched five-sample medians were:

The isolated rerun resolved earlier depth-1,024 variance, so SDK publish_many helpers cap requests at the protocol limit of 1,024 and split larger inputs sequentially. Callers can choose a smaller explicit batch for a tighter latency or partial-failure boundary; payload size and subscriber fanout still need workload-specific measurement.

Negotiated PubSub batch events

With the same 65,536 publishes per publisher, depth 1,024, 256-byte payload, and compact-mode-35 workload, three-sample medians compare the legacy event frames with pubsub_batch_v1 negotiated on every subscriber:

For a 1,024-message batch with 256-byte values, one subscriber's encoded output fell from 1,024 frames and 422,912 bytes to one frame and 267,428 bytes (-36.8%). The encoder result also fell from 67,622 retained heap words to 10 because the batch is held by one encoded binary. Ordinary single-publish memory was unchanged in the matched Benchee gate.

For homogeneous batches with eight matching pattern subscribers, resolving a complex pattern once per channel batch reduced the 1,024-message median from 5.64 ms to 3.05 ms (-45.9%) while retaining exact-before-pattern ordering for every message.

FerricStore Streams: local optimization baseline

These development numbers were recorded on an Apple M4 Max with a one-node Raft group and local durable WAL commits. They validate relative code-path changes; they are not multi-node product claims.

Targeted before/after gates during the Stream apply-plan cleanup measured the following on the same machine and process shape:

• publishing 1,024 ordered member-catalog rows fell from roughly 0.55 ms median to 0.33 ms after the committed plan began carrying ready catalog rows and an exact idempotent count;
• publishing the derived metadata/index cache for 64 topics fell from roughly 207 µs to 34-40 µs after terminal plans began carrying complete ready cache rows through the durable commit boundary;
• XRANGE COUNT 10 improved from about 94.7K requests/s at 9.21 µs median to about 168-170K requests/s at 4.79 µs median after range catalogs began using OTP ordered lookup/traversal, dropped unused LFU-hit bookkeeping, reused per-shard catalog references, fused the type check with the bounded page read, and assembled the common exact persisted row without an intermediate result;
• new-topic type validation fell from about 1.86 µs to 0.62 µs in isolation by validating the already-read marker, and preparing the type/entry/meta physical keys together reduced that encoding component by about 41%;
• staging 64 shared-log topics fell from 122 µs to 89 µs median in a 40-sample paired run after their validated plans were staged once. Empty optional index checks fell from about 29 µs per 64 topics to 0.25 µs, and empty blocking-waiter notification work fell from about 52-60 µs to 0.3 µs;
• in a 200-append steady-state projection microbenchmark, passing ready put-only rows directly to the existing mixed-operation Bitcask NIF reduced total BEAM allocation/GC plus append/result processing from about 339 ms to 192 ms (roughly 43%); the on-disk format and fsync boundary are unchanged;
• bounded activity-log publication for 1,024 results fell from 1,122 µs to 222 µs median in the focused microbenchmark because only the newest retained rows are materialized;
• constructing a 64-entry deterministic append plan fell from 9.63 µs to 7.25 µs median, while measured allocation fell from 34.40 KB to 30.42 KB, after each durable compound key became the single owning binary for its returned Stream ID slice;
• direct mixed-topic preparation uses one routing/grouping pass instead of allocating generic command tuples and rediscovering the Stream shape per shard. In the 64-entry/12-shard microbenchmark it used 22.27 KB instead of 28.34 KB; the eight-topic/same-shard shape used 32.87 KB instead of 38.51 KB. Timing gains were about 3-16% across paired samples, while the same-shard median remained within measurement noise;
• grouped-command uniqueness validation now performs one map insertion/hash per topic. The 64-entry gate improved from 1.90 to 1.77 µs for eight topics and from 13.01 to 12.42 µs for 64 topics without relaxing duplicate, gap, or out-of-range rejection;
• exact native request/response schemas reduced the common five-binary XRANGE request decode from about 409 ns to 300 ns and the ten-row response value encode from about 1.07 µs to 0.46 µs. Python and TypeScript response decoding also improved without regressing mixed arrays; Go uses the same preallocated output shape but could not be executed on the benchmark host because its Go toolchain was unavailable;
• matched native mode-34 producers reached about 401K entries/s over loopback TCP with 16 concurrent connections, while a single in-flight request remained latency-bound. This is the native counterpart to the concurrent embedded gate, rather than multiplying a sequential request rate into a throughput claim;
• with the corrected compact-command apply budget, validated internal scale samples reached about 326K p50-derived entries/s at batch 1,024, 300K at batch 2,048, and 174K at batch 4,096. Batch 1,024 was the current sweet spot on this host; producers should benchmark rather than assuming that the largest admitted batch wins.

These are optimization gates, not replacements for the public table: the absolute rerun was discarded because unrelated long-running processes were saturating several CPU cores. Relative gates were accepted only when compared inside one alternating run or isolated to one measured stage.

One measured experiment was explicitly rejected: carrying pre-encoded field payloads in the Raft command improved 1,024-entry median latency by about 4%, but enlarged the replicated term from 26.7 KB to 36.9 KB (about 38%). The branch keeps the compact field-list WAL shape rather than trading durable write and retention amplification for that small CPU gain.

The final sustained 1,024-entry topology gate used 32 batches, concurrency 16, three samples, 12 shards, 64 same-shard topics, and disabled optional activity logging. It verified exact final lengths for every physical Stream. The earlier single-request 1,024-entry append sample had 4.19 ms average and 3.95 ms p50 latency. At 256 entries, traced embedded requests were typically about 1.8 ms; roughly 0.5 ms was deterministic apply work and the remaining majority was the Raft/WAL commit wait, with occasional storage tails. This is why larger embedded producer batches cross 200K entries/s while smaller batches can report fewer entries/s even though the entry-processing code is the same. The native numbers additionally include loopback TCP framing, decoding, dispatch, and response encoding; their storage-tail deviation was high, so both mean throughput and p50-derived rate are shown.

Comparison boundary: Redis, RabbitMQ, and Kafka

Published headline rates are not directly comparable unless the acknowledgement and durability boundary is identical:

For headline context only, Redis documents roughly 0.5-1M pipelined XADD/s on an average machine, and RabbitMQ's Stream overview published about 1.2M confirmed messages/s on its example workstation. Neither acknowledgement is the same as FerricStore's local fsynced-WAL boundary: the Redis figure is persistence-policy dependent, and RabbitMQ Streams confirm after quorum replication without explicit fsync. Kafka deliberately scales by partition and producer batching, so a single context-free records/s number would be more misleading than useful.

Primary references: Redis Streams performance, Redis persistence, RabbitMQ Streams data safety, RabbitMQ Streams overview benchmark, RabbitMQ quorum queues, Kafka topic flush configuration, and Kafka replication design.

For an honest cross-system run, hold payload bytes, producer count, batch size, partition count, replica count, acknowledgement policy, fsync boundary, retention, consumer acknowledgements, warm-up, and measurement interval fixed. Report both batches/second and entries/second, plus p50/p95/p99 latency. A Flow workflow completion is another distinct unit: it can include multiple state, index, history, and read operations and must not be compared to one Stream entry or one batch request.

FerricFlow: latest Azure runs

Workload shape:

1,000,000 flows
single FerricStore server VM
single Python SDK client VM
Flow queue/workflow workers
live mode: create and process run together

The best balanced 16-vCPU server result was with 32 Flow shards.

Server CPU scale

These runs used default server behavior and live 1M-flow workloads.

16-vCPU shard sweep

Sync live runs:

Async live runs:

Interpretation: 32 shards was the best balanced setting in these Azure runs. 64 shards slightly improved queue-only sync throughput, but 32 shards was better for the workflow mix.

External Reference: DBOS Published Numbers

DBOS publishes Postgres-backed durable workflow benchmark numbers that are useful as an external reference point. These are not an apples-to-apples comparison with the FerricFlow tables above: DBOS used a single AWS RDS Postgres db.m7i.24xlarge instance with 96 vCPUs, 384 GB RAM, and 120K provisioned IOPS on io2 storage, while the FerricFlow numbers above are from separate Azure server/client VM runs with FerricStore's native storage engine.

Source: DBOS, "Does Postgres Scale?", April 23, 2026.

DBOS docs summarize the same scale as >40K workflows or steps per second for a DBOS application using one Postgres database: DBOS Architecture and DBOS Production Checklist.

Older DBOS workflow-latency benchmarks against AWS Step Functions are a different workload shape, but they give another public DBOS reference: DBOS vs. AWS Step Functions Performance Benchmark reports DBOS Transact as 25x faster than standard Step Functions in their tests, with a 5-step workflow around 40 ms in DBOS versus over 1 second in Step Functions, and Express Step Functions around 3x slower than DBOS.

KV SET/GET: native protocol baseline pending

The older KV benchmark shape is no longer valid because the standalone server now exposes the Ferric native binary protocol. Publish KV SET/GET numbers only after rerunning them through a native SDK or native protocol benchmark client.

The replacement benchmark should report at least:

Reproducing the shapes

FerricFlow benchmarks are run from the Python SDK repository with the optimized queue/workflow benchmark scripts. KV benchmarks should use a native-protocol SDK/client shape for current FerricStore releases.

For public reporting, prefer the 1M-flow live results. Add KV tables only after native-protocol SET/GET runs are available.