A per-session Docker stack — sandbox plus sidecars — over the Engine API.
Same job as CrowdControl.Provider.Docker, one container more, and one
property that provider provably cannot have: a structural egress block on
the sandbox that still leaves the agent reachable from the host.
There is no docker compose dependency and there must never be one. The
Engine API has no compose endpoints — compose is a client-side Go plugin that
synthesises exactly the calls below — so shelling out would buy a binary
dependency, a YAML round trip and a second error vocabulary in exchange for
nothing.
The network shape, and why it needs two containers
The measured constraint, confirmed six independent ways against a live daemon:
On one container,
Internal: trueand a published port are mutually exclusive. Publishing requires at least one non-internal endpoint, and attaching one restores full internet egress.
So the sandbox and the thing the host talks to cannot be the same container:
<project>-sbx—Internal: true. The sandbox sits here and only here. This is the one strong egress primitive: no default route exists at all, so the internet, the Docker host, containers on every other Docker network and Docker's own embedded DNS are unreachable structurally rather than by a missing NAT rule.<project>-pub— non-internal, soPortBindingsactually bind, but withcom.docker.network.bridge.enable_ip_masquerade=falseso the one container attached to it has no internet either.<project>-egress— a plain NAT bridge, created only if some sidecar declaresegress: :allow. If nothing asks, it never exists.
A forwarder sidecar is dual-homed on 1 and 2 and publishes the agent port
on 127.0.0.1:0. It runs
socat TCP-LISTEN:<port>,fork,reuseaddr TCP:<sandbox>:<port>, addressing the
sandbox by its network alias. fork is not decorative: a single-slot
forwarder (busybox nc -e) dropped two of three concurrent requests in
testing, and CrowdControl.Backend.Sandboxd holds a long-lived chunked
stream open for the whole session.
The forwarder's second network is attached with
POST /networks/{id}/connect after create, not by listing two endpoints in
POST /containers/create: create-time dual attach is HTTP 400 below API
1.44, and daemons still report MinAPIVersion 1.40. The connect-after-create
path is verified to produce an identical container.
Options
Stack:
:services— sidecar specs, each a map (see below). Default[].:sandbox_service— the name the sandbox container is given, default"sandbox". A spec in:serviceswith this name is the sandbox: it supplies the image, command, env and mounts, and the provider overlays the agent contract on it. With no such spec the sandbox is synthesised from:imagealone.:forwarder_service— the name the forwarder is given, default"forwarder". Always synthesised; see:forwarder_image.:forwarder_image— default"alpine/socat:1.8.1.3". Any image withsocatonPATHworks; the provider setsEntrypointitself.:network—[internal: true, driver: "bridge", options: %{}]. See the warning below.:volumes— named volume declarations,[%{name: "workspace"}]. Created as<project>-<name>and destroyed with the stack.:ready— per-service healthchecks,%{"db" => %{test: [...]}}.:project_name— default"cc-<session_key>".:proxy_service— the sidecar fronting the egress proxy. See below.:health_timeout— deadline for the healthcheck poll, default60_000.
Agent, exactly as CrowdControl.Provider.Docker takes them: :image,
:agent_port, :capture_path, :ready_timeout, :req_adapter,
:docker_host, :timeout.
Hardening (:cpus, :memory, :cap_drop, :security_opt, :pids_limit,
:user, :readonly_rootfs, :tmpfs) comes from these top-level options and
applies to every container in the stack, through
CrowdControl.Backend.Docker.HostConfig. There are deliberately no
per-service overrides: one posture per stack is the whole reason that module
exists, and a sidecar quietly weaker than the sandbox it shares a network
with is not a useful thing to be able to express.
Service spec
%{
name: "proxy", # required, unique, DNS-safe
image: "cc/egress-proxy:1.2.3", # required
egress: :allow, # required for sidecars, see below
entrypoint: ["/proxy"], # optional
command: ["--listen", ":8080"], # optional
env: %{"LOG_LEVEL" => "info"}, # optional
volumes: [%{name: "cache", target: "/cache", read_only: false}],
depends_on: ["db"], # optional, topologically ordered
user: "1000:1000", # optional
port: 8080 # only read for :proxy_service
}Every service gets its name as an alias on the sandbox network, so services
address each other by name and nothing has to learn an IP.
Posture is never inferred
:egresson a sidecar is required and has no default.:nonekeeps it on the internal network only;:allowalso attaches it to the NAT bridge. Omitting it is{:error, {:compose, {:egress_required, name}}}, for the same reasonCrowdControl.Backend.Dockerrefuses to guess:network_modeandCrowdControl.Provider.Dockerrefuses to guess:egress. A sidecar with:allowsits on both networks and can relay the internet into the sandbox — which is exactly what an egress proxy is for, and exactly why saying so is mandatory.- The sandbox service can never carry
:allow; asking is{:error, {:compose, :sandbox_egress_forbidden}}. network: [internal: false]is accepted, and gives the sandbox a NAT bridge and full internet. It is the one option here that throws away the module's reason to exist, so it exists only as an explicit, typed-out act.
Egress proxy
No proxy ships with this library; SECURITY.md specifies what a conforming
one must do. Naming one with :proxy_service wires both halves of that
contract:
- the sandbox's environment goes through
CrowdControl.Backend.Credentials.apply_credentials/2—ANTHROPIC_BASE_URLpoints at the proxy's alias andANTHROPIC_API_KEYbecomes a per-session token, with any real:api_keyremoved rather than overridden; - the proxy receives
CC_SESSION_TOKEN(the minted token, so it can recognise the session) and the realANTHROPIC_API_KEY(so it can substitute it upstream).
The proxy must declare egress: :allow, or it could not reach the upstream
API and the sandbox would fail in a way that looked like a model bug:
{:error, {:compose, {:proxy_needs_egress, name}}}.
With no :proxy_service, :api_key is placed in the sandbox's own
environment unchanged and no ANTHROPIC_BASE_URL is set. That is the honest
no-proxy posture — the sandbox holds a real provider credential — and it is
what makes the removal above observable rather than notional. :api_key and
:session_token are both in CrowdControl.Store.secret_keys/0, so neither
survives scrub/1 into a Store record.
Unlike CrowdControl.Backend.Docker, this provider needs no explicit
:network_mode to make the proxy enforcing. There is no bridge to
accidentally choose: the sandbox network is created by this module, is
internal by default, and lives and dies with the session.
The published port is never persisted
Every stop/start/restart allocates a new ephemeral host port, and
while a container is stopped NetworkSettings.Ports is {} rather than
reporting the old one. reconnect/1 therefore always re-reads the
forwarder's port. Measured behaviour, not caution.
Summary
Types
@type t() :: %CrowdControl.Provider.Compose{ agent_port: pos_integer(), capture_path: String.t(), config: keyword(), containers: [{String.t(), String.t()}], created_at: integer() | nil, egress_network: String.t() | nil, forwarder_service: String.t() | nil, owner: String.t() | nil, project: String.t() | nil, publish_network: String.t() | nil, sandbox_network: String.t() | nil, sandbox_service: String.t() | nil, session_key: String.t() | nil, volumes: [String.t()] }