ExSandbox.Conformance.Network (ExSandbox v1.0.1)

Copy Markdown View Source

Conformance group: network confinement (005 T060b, T060d; 003-FR-002, 005-FR-003, 005-FR-011aFR-011d).

Why this group did not exist, and what its absence cost

003-FR-002 says no sandbox may observe or connect to another. Until this group, the published contract did not check it. The only network assertion in the repository lived in test/mechanism/beam/isolation_cluster_test.exs — a mechanism-private test, shipped with ex_sandbox's own suite rather than with the contract a third party runs.

So a third-party mechanism could implement ExSandbox.Mechanism, pass the entire published suite, and provide no network isolation whatsoever. 012-FR-010's promise that the suite is "usable by any mechanism implementation" was true of the code and not of the guarantee.

Both directions, because denial alone is not a policy

The first version of this group would have checked one thing: that a sandbox cannot reach what it must not. That check is passed perfectly by a mechanism that permits nothing — no DNS, no API calls, no webhooks out — which is what --unshare-net gives today and is not what a tenant application can run under (005-FR-011a).

A denial-only suite therefore rewards the wrong mechanism: it scores a sandbox that cannot work at all above one that implements a real allowlist, because the second has to get the permitted half right and the first has no permitted half to get wrong. FR-011d requires both directions for this reason, and the checks below come in pairs.

⚠️ The permitted-direction check reports capability unavailable, not failure, when the mechanism declares no allowlist. A mechanism that denies everything is at an earlier point on the road, not in violation — but it must not be able to collect a green tick for the half it never implemented.

The transport must not be the subject

Every check reaches into the sandbox through context.exec, never through distribution. Asking "can this sandbox reach the platform?" over :erpc cannot answer: a refused connection and an unreachable sandbox are both :noconnection, so the suite cannot tell the boundary holding from the test never running.

This is not hypothetical tidiness. It is the measured failure that isolation_cluster_test.exs documents at length, and it is why that file reaches its sandbox over stdio.

Summary

Functions

Attempts to reach a destination the environment does not permit (005-FR-011c).

Attempts to reach the platform node's own listening socket.

Attempts to open a TCP connection from one sandbox to another.

Attempts to send UDP datagrams out of the sandbox's namespace (029-FR-013, SC-003, 029 T005).

Attempts to widen the environment's own allowlist from inside the sandbox (FR-011b).

Every handle by which the platform makes sandbox reachable from outside it (029-FR-017, 029 T034a).

The destination this run treats as permitted, and the one it treats as denied.

The destination this run treats as permitted and names by hostname (029-FR-012, 029 T004).

Requires that every declared handle of other is refused from inside sandbox, and that at least one of them was actually exercised (029-FR-017).

Requires that a permitted destination named by hostname is reachable (029-FR-012).

Requires that a permitted destination is actually reachable (FR-011a, FR-011d).

The context the group's sandboxes are built with.

Emits the network checks into the calling test module.

Functions

attempt_reach_denied_host(mechanism, sandbox)

Attempts to reach a destination the environment does not permit (005-FR-011c).

⚠️ Gated on a host control probe against this same address, and the shape of that control is the whole correctness of this check.

The address is documentation-reserved (RFC 5737 TEST-NET-1), chosen because it is nominally routable rather than a black hole, so that a refusal would be evidence of policy. That reasoning does not survive contact with the internet.

Measured on a macOS dev host: nc -z -w 3 203.0.113.1 443 ignored its own bound and blocked for 75 seconds, so the suite's outer timeout 3 fired first and produced exit 124. Exit 124 is TIMEDOUT, which this group deliberately scores as a refusal — a probe that ran and saw no answer within its bound is exactly what a DROP policy looks like from inside.

Every step is right in isolation, and together they made this check pass against ExSandbox.PorousMechanism: a mechanism that runs every command in the host's own unconfined shell and has no policy whatsoever.

The first fix attempted here was a general reachability baseline — can the host reach the open internet at all? It measured yes on this host, and the check went on passing. That refuted the diagnosis: TEST-NET-1 is nominally routable but nobody actually routes it, so it is silent for everyone regardless of host health. A liveness question about the host cannot detect that, because the host is fine.

So the control probes this exact address, from the host, outside any sandbox. If the host — subject to no sandbox policy — also sees silence, then silence from inside the sandbox distinguishes nothing, and the check reports the third outcome. It has force only where the host reaches the address and the sandbox does not: the one configuration where the difference is attributable to the boundary.

012-FR-016a exists for precisely this: a check that cannot be demonstrated here is not a check that passed.

attempt_reach_platform(mechanism, sandbox)

Attempts to reach the platform node's own listening socket.

attempt_reach_sandbox(mechanism, sandbox, other)

Attempts to open a TCP connection from one sandbox to another.

⚠️ Addressed by IP and port, not by node name. An undistributed sandbox has no node name, so a name-based attempt fails for a reason that has nothing to do with the network boundary and reports a pass either way.

attempt_udp_egress(mechanism, sandbox)

Attempts to send UDP datagrams out of the sandbox's namespace (029-FR-013, SC-003, 029 T005).

Two legs, both named by SC-003: the host's loopback as the sandbox sees it, and the namespace's gateway. The gateway leg is the one with force -- under pasta that address is the namespace's own resolver, so an answer from it is a datagram that left and came back, which no TCP probe in this group can observe because the redirect is meta l4proto tcp.

⚠️ Requires the mechanism to declare context.udp_probe. Without it this reports the third outcome rather than guessing, for the same reason attempt_widen_allowlist/2 does: there is no host-neutral way to send a datagram from inside someone else's namespace, and a shell probe that could not run would score "we did not try" as "the boundary held".

attempt_widen_allowlist(mechanism, sandbox)

Attempts to widen the environment's own allowlist from inside the sandbox (FR-011b).

The hostile act is reconfiguration, not connection: tenant code that can edit the policy does not need to defeat it.

Requires the mechanism to declare context.policy_handle. Without it the check reports the third outcome rather than guessing — see the implementation for the two guesses that passed against a mechanism with no policy at all.

denied_address()

@spec denied_address() :: {String.t(), pos_integer()}

peer_handles(arg1)

@spec peer_handles(ExSandbox.Sandbox.t() | map()) :: [term()]

Every handle by which the platform makes sandbox reachable from outside it (029-FR-017, 029 T034a).

⚠️ A declaration the mechanism owes the suite, not something the suite can discover. FR-011e already established that shape for context.address, and the reason is stronger here: the suite cannot enumerate a mechanism's publishing surfaces. It cannot know that a container runtime also answers on a bridge address and a service name, or that this platform will shortly publish a host-side tuple and a public hostname for the same sandbox. So the set is declared, and a handle the platform publishes but the mechanism does not declare is ungoverned by construction.

Read from context.peer_handles — a list of {host, port} — falling back to [context.address] when that is already a dialable tuple, so a mechanism that publishes exactly one handle declares nothing extra.

⚠️ Mechanism-neutral on purpose (D27). Nothing here names a namespace, a port forward, a table, or a bridge. A container mechanism declares its published port and its runtime-resolvable name; this one declares its netns tuple and, after 029 T033, its host-side tuple. The rule survives the transfer that D27 says T031 and T032 do not.

permitted_address()

@spec permitted_address() :: {String.t(), pos_integer()}

The destination this run treats as permitted, and the one it treats as denied.

⚠️ Both are configuration, not constants, because a check gated on reachability cannot hardcode what is reachable. attempt_reach_denied_host/2 reports the third outcome wherever the host cannot reach the denied address itself, so a deployment whose egress differs from the defaults supplies its own pair rather than recording a permanent gap:

config :ex_sandbox, :conformance,
  permitted_destination: {"example.internal", 443},
  denied_destination: {"blocked.internal", 443}

They must not overlap, and denied_address/0 refuses if they do -- an allowlist containing the denied destination would make FR-011c fail against a mechanism doing exactly what it was told.

permitted_name_address()

@spec permitted_name_address() :: {String.t(), pos_integer()}

The destination this run treats as permitted and names by hostname (029-FR-012, 029 T004).

Configurable for the same reason permitted_address/0 is -- a deployment behind an egress proxy names a host it can actually resolve and reach:

config :ex_sandbox, :conformance,
  permitted_name_destination: {"api.internal", 443}

require_every_peer_handle_refused(mechanism, sandbox, other)

@spec require_every_peer_handle_refused(
  module(),
  ExSandbox.Sandbox.t(),
  ExSandbox.Sandbox.t()
) :: :ok

Requires that every declared handle of other is refused from inside sandbox, and that at least one of them was actually exercised (029-FR-017).

The three outcomes, and why the middle one is not a pass

Per handle:

  • the platform cannot reach it → not exercised. A refusal from inside is not evidence when the destination answers nobody: EditablePolicyMechanism publishes an address with no listener behind it, and the existing peer check passes against it for that reason alone.
  • the platform reaches it and the sandbox does not → refused, which is the only thing that counts as evidence.
  • the sandbox reaches it → crossed, which fails outright.

Aggregated: any crossing fails; otherwise any refusal passes; otherwise the third outcome, naming every handle and why each was not exercised.

⚠️ The order is chosen. "Any crossing fails" beats "any refusal passes" so a mechanism cannot buy a green tick for one handle while leaking through another — which is the entire difference between this check and the one that probes context.address alone.

⚠️ A mechanism declaring no handle reports the third outcome, never a pass. That is the state today: FR-017 holds because a sandbox's address is unnameable, so there is nothing to attempt, and "the boundary was not exercised" is the true statement about it. Scoring it green would be FR-015's control that reads as the guarantee it is not.

require_permitted_name_reachable(mechanism, sandbox)

Requires that a permitted destination named by hostname is reachable (029-FR-012).

⚠️ Three gates before the probe runs, and each of them exists because the obvious version of this check reports something it did not measure:

  1. The mechanism must declare an allowlist at all. Without one there is no policy for a name to be matched against, and dialling anyway would score a mechanism that denies everything.
  2. The host must itself resolve and reach the name. Same control as attempt_reach_denied_host/2: if the host is silent too, silence from inside distinguishes nothing.
  3. The name must not resolve to an address the allowlist already carries as a literal. This is the one that is specific to name matching. If the hostname's address is the IP entry the suite also permits, then a successful connection is explained by the literal and the name was never consulted -- a green tick for a capability that does not exist.

require_permitted_reachable(mechanism, sandbox)

Requires that a permitted destination is actually reachable (FR-011a, FR-011d).

⚠️ Reports capability unavailable rather than failure when the mechanism declares no allowlist. A mechanism that denies everything has not violated FR-011a so much as not reached it — but the distinction has to be visible, because a silent skip here is what lets a deny-everything mechanism collect a full green suite while being unable to run any tenant application.

suite_context()

@spec suite_context() :: map()

The context the group's sandboxes are built with.

⚠️ Opt-in, and the opt-in is a measured admission rather than caution.

Carrying a network_allowlist is what makes the four checks in this group real: :permitted is derived from it, so without one require_permitted_reachable/2 has nothing to dial and every check reports the third outcome. The allowlist is therefore the whole point — and it is off by default because on the BEAM mechanism it does not yet work.

Measured (isolation run 2026-08-18T09:37Z-97, with the allowlist unconditionally on): all five checks FAILED with could not provision: {:error, :mechanism_error}, because a populated allowlist makes NodeLauncher take the policed branch, and that branch cannot currently boot a tenant:

Couldn't write to /proc/self/uid_map: Operation not permitted
setpriv: setresuid failed: Invalid argument
[error] sandbox node failed to boot: {:boot_failed, {:exit_status, 127}}

pasta in spawn mode creates a user namespace it cannot map, so every process inside is uid 65534 and setpriv --reuid fails for any uid. See egress-path-measurements.md.

⚠️ Why this was a flag and not a silent revert. The failures above were honest — the mechanism genuinely could not build the boundary, and the suite was right to refuse to report one. But they aborted the credentials phase before the isolation phase ran at all, so leaving them on cost every other measurement in the census and gave nothing back. The flag kept the capability one line away and kept the reason written down, rather than deleting the work and rediscovering it.

The default is now ON (T060a4e resolved)

The blocker above is fixed: LaunchPlan.build/4 inserts pasta after the privilege drop rather than wrapping the command in it, so setpriv runs in the host's fully mapped namespace and the tenant boots. Both halves of that ordering were measured, not argued (docker/launch-ordering-probe.sh): pasta composes after the drop (uid_map = 0 <uid> 1), and the scope's MemoryMax still SIGKILLs a 192MB allocation at a 64M cap across two intervening execs.

With the launch bootable, the trade reverses. Off is now the setting that costs measurement: the checks report the third outcome forever, and the permit direction — the only half that can distinguish an allowlist from blanket denial — is never exercised. The flag remains, because turning it off is how a host that genuinely cannot police egress keeps the rest of the census.

⚠️ It is deliberately NOT an exclusion (012-FR-011). Off, the checks report capability_unavailable — visible in the census, counted against the baseline, and impossible to mistake for a pass. A mechanism gets no credit for the checks it does not run.

config :ex_sandbox, :conformance, allowlist_enabled: false

tests()

(macro)

Emits the network checks into the calling test module.