ExSandbox.Mechanism.Beam.NodeLauncher (ExSandbox v1.0.0)

Copy Markdown View Source

Starts, probes, and terminates one hardened sandbox node (005 T020-T026).

:peer is the launcher, not the boundary

:peer.start_link/1 starts an OS-level BEAM node and gives us a supervised handle on it. That is all it gives us. It does not confine the filesystem, drop privileges, cap memory, or clear the environment — every one of FR-007 - FR-011 is satisfied by the command in exec, built by ExSandbox.Hardening, and by nothing in this module.

Stated plainly because the option map below reads like configuration, and it would be easy to add an option here believing it tightened something. The two options that actually carry weight are exec (the boundary) and the per-sandbox -setcookie (FR-003); the rest bound failure.

Summary

Types

What a caller needs to address and later reclaim a launched node.

Functions

Where the namespace acceptor helper lives.

The host pid of the confined BEAM below root_pid.

The allowlist a sandbox was provisioned with, or [].

Rebuilds {prog, args} from a plan's pasta_command.

The host's pid for a sandbox spawned through port.

Turns an allowlist into a policed command, or refuses the launch.

Launch one hardened sandbox node.

Installs the redirect into a launched tenant's namespace.

Application-level liveness (FR-021).

Terminates a sandbox's acceptor, idempotently (003-FR-013).

Stop a sandbox node, releasing all its memory including its atom table (FR-006).

Types

launched()

@type launched() :: %{
  node: node(),
  os_pid: integer(),
  peer: pid(),
  cookie: atom(),
  binding: ExSandbox.Egress.Binding.t() | nil,
  acceptor_os_pid: integer() | nil
}

What a caller needs to address and later reclaim a launched node.

Functions

acceptor_helper_path()

@spec acceptor_helper_path() :: String.t()

Where the namespace acceptor helper lives.

Public because a release that ships without it produces a launch failure whose cause is a missing file, and naming that file is the difference between a one-line fix and a namespace investigation.

confined_beam_pid(root_pid)

@spec confined_beam_pid(pos_integer()) :: {:ok, pos_integer()} | {:error, term()}

The host pid of the confined BEAM below root_pid.

⚠️ The port's own pid is not the thing to verify. The command is a chain -- systemd-run execs setpriv execs bwrap -- and bwrap forks: an outer process stays in the origin's namespaces to supervise, and only its child enters the new ones. So the pid the port reports is the outer supervisor, whose /proc/<pid>/ns/mnt and ns/net are identical to ours.

That made verify_applied/1 return :not_applied for a sandbox that was in fact fully confined -- measured: the port's pid reported the origin's namespaces, while its grandchild beam.smp reported mnt:[4026533087], net:[4026533091], uid 117068, memory.max 268435456, and cpu.max 50000 100000, all correct.

This is the same class of mistake as reading :os.getpid() from inside the sandbox, one level out: a pid that is correct in one frame of reference used in another.

Why beam.smp specifically, and not "a confined descendant"

Selecting the first descendant that looks confined would accept a short-lived helper while the actual sandbox ran unconfined -- the check would pass for a process nobody cares about. The sandbox is the BEAM, so that is what must be named. Exactly one beam.smp exists in the chain; more than one means the tree is not the shape this reasoning assumes, and guessing between them is precisely what must not happen, so it is an error rather than a choice.

egress_allowlist(sandbox)

@spec egress_allowlist(ExSandbox.Sandbox.t()) :: [
  ExSandbox.Egress.Policy.destination()
]

The allowlist a sandbox was provisioned with, or [].

Public because it is a decision -- whether this sandbox gets a policy at all -- and launch/2 cannot run on any host that is not Linux. Left private, the rule would be verifiable only where the whole launch works, which is the arrangement that let the earlier context-discard defect survive: the allowlist was parsed correctly, then dropped, and every test asserted on the parse.

exec_from_plan(launch_plan)

@spec exec_from_plan(ExSandbox.Egress.LaunchPlan.t()) :: {String.t(), [String.t()]}

Rebuilds {prog, args} from a plan's pasta_command.

⚠️ Always the plan's own head, never the original program. Keeping the original while taking the plan's arguments produces a command that reads correctly in a log line and execs the wrong binary -- and because the arguments still contain every hardening flag, a test that greps the joined string for --unshare-net or the scope name would pass.

⚠️ Which binary that head names changed with the split ordering, and the change is invisible here on purpose. pasta used to wrap the whole command, so the head was pasta; it is now inserted after setpriv, so the head is systemd-run once more. Reading the head off the plan rather than deciding it here is what let that move without this function knowing -- and a version that hardcoded either name would have been correct when written and silently wrong afterwards.

host_os_pid(port)

@spec host_os_pid(port()) :: {:ok, pos_integer()} | {:error, term()}

The host's pid for a sandbox spawned through port.

⚠️ Read from the port, never by asking the sandbox. The sandbox runs under --unshare-pid, so :os.getpid() inside it returns the namespace-local pid (2) while the host knows it by something else entirely (1565 in the run that found this). verify_applied/1 reads /proc/<pid> on the host, so the sandbox's own answer sends it to an unrelated process and it reports :unverifiable for a perfectly confined sandbox.

Taking it from the port also means verification needs no cooperation from the thing being verified: a compromised sandbox cannot misreport its pid to escape the check.

install_policy(exec, allowed, acquire \\ &ExSandbox.Egress.Binding.acquire/1)

@spec install_policy([ExSandbox.Egress.Policy.destination()], term(), (list() ->
                                                                   term())) ::
  {:ok, term(), term(), term()} | {:error, atom()}

Turns an allowlist into a policed command, or refuses the launch.

Public for the same reason as egress_allowlist/1: this is a decision -- whether a tenant who cannot be given a policy is given a sandbox anyway -- and launch/2 runs on Linux and nowhere else. Left private, the rule would be verifiable only on a host where the whole launch works.

⚠️ The rule is refuse, never downgrade, and the downgrade is the dangerous branch rather than the obvious one. A host that cannot supply a binding could fall back to --unshare-net and produce a sandbox that reaches nothing, which passes every denial check in the network group while the census reports the group as demonstrated -- 005 T060a5's named false pass, and the same shape as the --unshare-net trap this whole feature exists to close.

acquire is injectable so the refusal can be provoked without exhausting a real pool; it defaults to ExSandbox.Egress.Binding.acquire/1.

launch(sandbox, opts \\ [])

@spec launch(
  ExSandbox.Sandbox.t(),
  keyword()
) :: {:ok, launched()} | {:error, atom()}

Launch one hardened sandbox node.

Refuses when the host hardening module reports it is unavailable (R9, Principle II): a host that cannot confine must not run tenant code unconfined. Returns {:error, :mechanism_error} after recording why, so an operator reads the missing capability rather than a bare atom.

police(plan, opts \\ [])

@spec police(
  ExSandbox.Egress.LaunchPlan.t(),
  keyword()
) :: {:ok, integer() | nil} | {:error, atom()}

Installs the redirect into a launched tenant's namespace.

⚠️ This runs after the tenant is already executing, and that is forced by the mechanism rather than chosen: the namespace does not exist until pasta creates it, and pasta creates it by starting the tenant inside it. There is a real window in which the tenant runs unpoliced.

The window fails closed. Until the redirect lands there is no NAT rule sending the tenant's traffic anywhere, and the acceptor it would be sent to is not listening -- so a connection attempted in the window reaches nothing. That is asserted rather than assumed: a failure here terminates the tenant, because the alternative is a running sandbox whose allowlist is not enforced, and which passes every denial check in the conformance suite.

Public because it is the ordering rule that cannot be exercised off Linux -- left private it would be verified only where the whole launch works, which is the arrangement that let the context-discard defect survive.

probe(peer)

@spec probe(pid()) :: :ok | {:error, :unresponsive | :down}

Application-level liveness (FR-021).

Separate from process monitoring because :unresponsive — process alive, control channel up, application wedged — is invisible to :peer (R6). A hung sandbox reported healthy is worse than a crashed one: it holds memory, blocks placement, and serves nothing.

⚠️ Takes the peer pid, not the node name, because the probe rides the stdio control channel. An earlier version called :erpc.call(node, ...), which cannot reach a sandbox running under --unshare-net — it has no network interfaces — so every healthy sandbox answered {:error, :down} and status/1 reported :absent.

That was the most dangerous form of this bug: reconciliation treats :absent as "reclaim it", so the better the network isolation worked, the more certainly a live tenant would be torn down as an orphan. Measured: two sandboxes that had just provisioned successfully, with nothing else running, both reported :absent immediately.

read_parent_oom_kills(cgroup_path)

stop_acceptor(os_pid)

@spec stop_acceptor(integer() | nil) :: :ok

Terminates a sandbox's acceptor, idempotently (003-FR-013).

⚠️ Exists because the acceptor does not die with the namespace it serves. Measured: after kill -9 on the namespace holder the acceptor was still running, and /proc/<acceptor>/ns/net still named the same namespace — so it was holding a dead netns open. A destroy that forgot it would leak one process and one namespace per sandbox, with no symptom until the host ran out of either.

Killing a pid that is already gone is not an error: a second destroy/1 must be safe, and reclamation that fails on an already-reclaimed sandbox is reclamation nobody can retry.

terminate(peer, timeout \\ nil)

@spec terminate(pid(), timeout()) :: :ok

Stop a sandbox node, releasing all its memory including its atom table (FR-006).

Always returns :ok: an already-dead node is success, matching 003's idempotency rule. Escalates to an OS kill after the timeout so a wedged sandbox cannot block reclamation — a sandbox that ignores a graceful stop is precisely the one that must not be able to hold capacity forever.