ExSandbox.Conformance.Helpers (ExSandbox v1.0.0)

Copy Markdown View Source

The shared vocabulary of the conformance suite (012 T033, T034, T034b).

Three things live here rather than in the individual groups, because all three are places where a plausible-looking implementation quietly stops measuring anything:

  • guarantee_failure/2 — every failure names the 003 requirement it violated, not merely the assertion that tripped (T033).
  • capability_unavailable/2 — the third outcome, distinct from pass and fail (T034).
  • demonstrate_breach/3 — the rule that a cap which cannot be shown stopping something is unavailable, never satisfied (T034b).

Summary

Functions

Asserts condition, naming requirement if it does not hold.

A sandbox struct for the suite's own use.

The third outcome: this host cannot provide what the check needs (T034).

Runs attempt — which must breach a cap — and requires the breach be stopped.

Raises a failure naming the violated 003 requirement.

Reports whether capability is available, as ExSandbox.Capability sees it.

Provisions a sandbox, converting a host's inability to confine into the third outcome rather than a failure.

Runs attempt — a hostile act — and requires that it failed.

Functions

assert_guarantee(condition, requirement, detail)

@spec assert_guarantee(term(), String.t(), String.t()) :: :ok

Asserts condition, naming requirement if it does not hold.

build_sandbox(overrides \\ [])

@spec build_sandbox(keyword()) :: ExSandbox.Sandbox.t()

A sandbox struct for the suite's own use.

Values are unremarkable on purpose; ExSandbox.OpacityTest is where hostile owner_ref shapes are exercised.

capability_unavailable(capability, detail)

@spec capability_unavailable(ExSandbox.Capability.name(), String.t() | nil) ::
  no_return()

The third outcome: this host cannot provide what the check needs (T034).

Not a pass and not a failure. Reported through ExUnit's skip mechanism so it shows up in the run rather than vanishing, and phrased so it cannot be mistaken for a green result.

Not requestable by the consumer. Every call site derives its argument from ExSandbox.Capability at runtime. If a configuration option could reach here, it would be an exclusion wearing another name and FR-011 would be violated.

demonstrate_breach(capability, requirement, attempt)

@spec demonstrate_breach(
  ExSandbox.Capability.name(),
  String.t(),
  (-> {:breached, term()} | {:stopped, term()} | term())
) :: :ok

Runs attempt — which must breach a cap — and requires the breach be stopped.

This is FR-012a and FR-012b in one function, and the shape matters more than it looks:

  • attempt returns {:breached, evidence} if the hostile act succeeded, or {:stopped, evidence} if the mechanism prevented it.
  • {:breached, _} fails, naming requirement.
  • {:stopped, _} passes.
  • Anything else — including the attempt erroring for an unrelated reason — routes to capability unavailable, never to a pass (FR-012b).

That last clause is the whole point. 005 R9b's composition configures a 100 MB cap and lets a process allocate 300 MB while exiting 0. A check that treated "we could not tell" as satisfied would report it conformant.

guarantee_failure(requirement, detail)

@spec guarantee_failure(String.t(), String.t()) :: no_return()

Raises a failure naming the violated 003 requirement.

SC-004 asks that a third-party mechanism author can act on a failure. An ExUnit assertion message tells them a comparison failed; it does not tell them which guarantee of the sandbox contract they have not met, and those are the terms in which the fix is expressed.

host_capability(capability)

@spec host_capability(ExSandbox.Capability.name()) :: ExSandbox.Capability.t()

Reports whether capability is available, as ExSandbox.Capability sees it.

provision_or_report(mechanism, sandbox)

@spec provision_or_report(module(), ExSandbox.Sandbox.t()) :: ExSandbox.Sandbox.t()

Provisions a sandbox, converting a host's inability to confine into the third outcome rather than a failure.

Why this is a helper rather than a pattern each check repeats

A mechanism that refuses to provision because the host cannot confine is behaving correctly005 R9 and Principle II require exactly that. A check that reports it as a conformance failure is blaming the mechanism for obeying the contract, and a run full of those failures is indistinguishable from a mechanism that is genuinely broken.

Two checks in lifecycle already did this translation and the rest did not, so the same host produced "capability unavailable" or a MatchError depending on which check ran. Measured on macOS: 29 of 33 checks reported failures for a mechanism doing the right thing.

⚠️ Not an exclusion (FR-011). The consumer cannot request this: it triggers only on the mechanism's own runtime refusal, carries the capability the host is missing, and is reported rather than silently passed. A mechanism that provisions successfully gets no benefit from it whatsoever.

require_refused(requirement, attempt)

@spec require_refused(String.t(), (-> {:refused, term()}
                                | {:succeeded, term()}
                                | term())) :: :ok

Runs attempt — a hostile act — and requires that it failed.

The isolation group's counterpart to demonstrate_breach/3. Same rule: the act must actually be performed, and {:refused, _} is the only pass.

A test asserting merely that no leak was observed passes against a mechanism with no isolation whatsoever, because nothing went looking.