ExSandbox.Conformance.Execution (ExSandbox v1.0.0)

Copy Markdown View Source

Conformance group: running a command inside a sandbox (008 T005).

Every limit check breaches, and nothing here inspects configuration

003-FR-012a is the rule and 005 R9b is the reason:

taskpolicy -m 100 sandbox-exec -f profile.sb ./hog 300

allocates 300 MB under a nominal 100 MB cap and exits 0. The limiter was invoked, with the right flag and the right number, and the cap is silently lost across the intervening exec. So a check that asserted execute/3 passed the cap along, or that the launch names it, or that the command exited cleanly, would certify that composition as conformant.

Each check here therefore runs a hostile command through the seam and requires the mechanism to show it was stopped. An attempt that neither completed its hostile act nor was visibly stopped is the third outcome — host capability unavailable — never a pass (FR-012b).

The check that is not about a limit at all

execute/3 against a destroyed sandbox must answer {:error, {:could_not_run, _}}. A mechanism that answers with a non-zero exit status instead has converted an unperformed check into a failed one, and 008-FR-026 says a failed check spends a refinement iteration that an unperformed one must not. That defect is invisible from every layer above: both shapes are "the verification did not pass", and only here can the two be told apart — which is why the check lives in the conformance suite rather than in 008's own tests.

Separation and truncation are checked by producing them, not by reading a doc

A command writes a known string to stderr and a different one to stdout, and the check requires each to arrive in its own field. 015 R17 measured MuonTrap's :logger_fun corrupting lines past a 256-byte buffer, so the long-line check emits a single line an order of magnitude past that and requires it back byte-identically: a mechanism whose capture reframes output into lines fails here rather than in someone's build log.

Summary

Functions

Runs a command, destroys the sandbox, and runs the same command again.

Requires a single line past any plausible buffer to survive intact (015 R17).

Runs a trivial command and requires the mechanism to report the command's own exit status.

Requires stdout and stderr to be attributable, by producing both.

Requires truncation to be declared, by producing more output than the mechanism will keep.

Runs one hostile command through execute/3 and classifies what happened.

Decides whether a breach through execute/3 was stopped, succeeded, or showed nothing.

Emits the execution checks into the calling test module.

Functions

assert_could_not_run_when_destroyed(mechanism)

Runs a command, destroys the sandbox, and runs the same command again.

The whole of 008-FR-016/FR-026 rests on the second answer being {:error, {:could_not_run, _}} rather than a non-zero exit status — and on the first one succeeding, without which the second proves nothing.

assert_long_line_intact(mechanism)

Requires a single line past any plausible buffer to survive intact (015 R17).

assert_runs(mechanism)

Runs a trivial command and requires the mechanism to report the command's own exit status.

⚠️ The status asserted is not zero. A mechanism that mapped every failure onto a non-zero exit would pass a zero-status check, and a mechanism that mapped every non-zero status onto an error would pass it too. Asking for a specific, unusual, non-zero status is what distinguishes "the command ran and this is its answer" from either.

assert_streams_separated(mechanism)

Requires stdout and stderr to be attributable, by producing both.

assert_truncation_declared(mechanism)

Requires truncation to be declared, by producing more output than the mechanism will keep.

⚠️ The assertion is not "output was truncated" — a mechanism with a generous limit legitimately truncates nothing. It is that truncated? and the bytes returned agree: if bytes were dropped the flag says so. Silent truncation of a build log is how a real error disappears from a diagnosis.

breach(mechanism, sandbox, command, dimension)

Runs one hostile command through execute/3 and classifies what happened.

Returns {:stopped, evidence}, {:breached, evidence} or an inconclusive value that ExSandbox.Conformance.Helpers.demonstrate_breach/3 routes to the third outcome.

classify(reason, dimension, cap, live)

@spec classify(term(), :memory | :cpu | :time, term(), {module(), term()} | nil) ::
  {:stopped, String.t()} | {:breached, String.t()} | {:inconclusive, String.t()}

Decides whether a breach through execute/3 was stopped, succeeded, or showed nothing.

Public because the group self-checks it in both directions: this is the single place a fail-open would live, so it is exercised directly rather than only through a live mechanism.

tests()

(macro)

Emits the execution checks into the calling test module.