Testing

Barrel P2P ships three layers of tests, in order of increasing realism:

• EUnit and CT suites that run inside a single BEAM. Property tests, gen_server fixtures, small two-node CT cases driven by ct_slave or peer. No docker, no external network.
• Local multi-node CT suites that spawn slave BEAMs on the same host and exercise the full dist handshake, including Ed25519 authentication and the -proto_dist barrel_p2p boot path. Still no docker.
• Docker e2e suites that bring up a multi-container cluster on a bridged docker network, with the full transport, Ed25519 authentication, and orchestrated startup ordering.

This document explains what each layer covers, how to run it, and where to look when something fails.

Quick reference

A clean rebar3 check reports Passed N tests. Skipped K (K, 0) tests. where K accounts for the docker-only suite and the gated soak suite, both of which print a tc_user_skip and exit cleanly.

EUnit and property tests

These live under test/*_tests.erl (EUnit) and test/*_prop_tests.erl (proper). They are fast and have no external dependencies.

rebar3 eunit

The interesting ones for understanding the system:

To run a single module:

rebar3 eunit --module=barrel_p2p_ormap_prop_tests

Local CT suites

These live under test/*_SUITE.erl. Most are single-node CT suites that exercise one module's gen_server behaviour. A few are multi-node:

To run one suite:

rebar3 ct --suite=test/barrel_p2p_dist_basic_SUITE

To run a single case:

rebar3 ct --suite=test/barrel_p2p_proto_dist_SUITE --case=two_node_connect

CT logs land in _build/test/logs/. The HTML index is _build/test/logs/index.html.

Soak suite

BARREL_P2P_CT_SOAK=1 rebar3 ct --suite=test/barrel_p2p_soak_SUITE.erl

One active case (broadcast_burst): five nodes, 40 plumtree broadcasts at 50 ms intervals, asserts every subscriber receives every marker. Useful as a regression check on the gossip layer.

Other scaffolding cases (partition-and-heal, cross-active-view-bang) live in the suite but are commented out of all/0; they have proven flaky under QUIC dist and we run them only manually.

Docker e2e

./docker/scripts/run_auth_tests.sh

What happens:

1. The script builds the multi-stage Docker image (docker/Dockerfile).
2. docker compose -f docker/docker-compose-auth.yml up brings up three cluster nodes (node1, node2, node3) with auth_enabled=true and auth_trust_mode=tofu, and a test_runner container.
3. The test_runner waits for the cluster to be healthy (/tmp/barrel_p2p_ready marker on each node), then runs barrel_p2p_docker_auth_SUITE.
4. The script exits with the test_runner's exit code; the stack is torn down.

What the suite covers:

• Three-node TOFU bootstrap: each peer pins the others on first contact.
• The Ed25519 challenge-response running through the upstream quic_dist_auth callback.
• Trust persistence across restarts.
• Whitelist (cookie_only_nodes) pattern matching.
• Service registration and RPC across the cluster.
• Cluster reformation after a node restart.

Flags:

• --no-build reuses the existing image (faster after a code change confined to test files).
• --cleanup tears down containers, networks, and volumes from a previous run.

CT logs from the docker run land in test_results/ on the host (via a bind mount).

Reading the results

test_results/index.html is the top-level CT report. test_results/all_runs.html lists every run; click into a specific one to see the per-case HTML.

The script's own exit code is 0 for green, non-zero on any failure. CI consumes the exit code; the HTML is for humans.

Interactive testing

For poking at a running cluster by hand:

ERL_AFLAGS="-proto_dist barrel_p2p -epmd_module barrel_p2p_epmd -start_epmd false" \
rebar3 shell --config config/sys.config --sname node1

In another terminal:

ERL_AFLAGS="-proto_dist barrel_p2p -epmd_module barrel_p2p_epmd -start_epmd false" \
rebar3 shell --config config/sys.config --sname node2

On node2:

1> barrel_p2p:join('node1@yourhost').
ok
2> barrel_p2p:active_view().
['node1@yourhost']
3> rpc:call('node1@yourhost', barrel_p2p, active_view, []).
['node2@yourhost']

inet:gethostname/0 tells you the short hostname; both shells need to resolve to the same yourhost.

Useful inspection:

%% Cluster + transport
barrel_p2p:active_view().
barrel_p2p:passive_view().
nodes().
sys:get_state(barrel_p2p_hyparview).

%% Trust + auth
barrel_p2p_dist_keys:list_trusted().
barrel_p2p_dist_keys:get_trust_mode().
{ok, Pub} = barrel_p2p_dist_auth:get_public_key().
barrel_p2p_dist_keys:fingerprint(Pub).

%% Process view
observer:start().

To run without auth (for faster local poking):

ERL_AFLAGS="-proto_dist barrel_p2p -epmd_module barrel_p2p_epmd -start_epmd false -barrel_p2p auth_enabled false" \
rebar3 shell --sname node1

Recovery

rebar3 ct reports '<some>_SUITE cannot be compiled or loaded' for a suite you have not touched: probably stale .erl files left in _build/test/lib/barrel_p2p/test/ from a different branch. Wipe the test profile and try again:

rebar3 clean -a
rm -rf _build/test
rebar3 ct

peer:start boot-timeout errors during a CT run usually mean the parent's code path was not propagated to the spawned BEAM. The convention used by the suites is to pass -pa flags built from code:get_path/0 explicitly; check the suite's start_peer/3 helper if you see this.

Adding your own test

For a new EUnit module, create test/<name>_tests.erl and add test functions ending in _test/0 or _test_/0. Properties go in test/<name>_prop_tests.erl using the proper macro form shown in the existing files.

For a CT suite, create test/<name>_SUITE.erl with the standard all/0, init_per_suite/1, etc. Multi-node CT cases should reuse the start_peer/3 helper from barrel_p2p_proto_dist_SUITE (which is -compile([export_all]) specifically to be reusable across suites).