Every kubereq call in the project, and the error vocabulary built on top of it.
This module plays the same role for CrowdControl.Backend.Kubernetes that
CrowdControl.Backend.Docker.API plays for the Docker backend: one file to
audit when the client library moves. That confinement earns more here than it
does there, because kubereq 0.4.4 is young and has sharp edges the backend
must never see:
Kubereq.list/3withinto: :streamsilently applieslimit: 10and silently truncates on a mid-pagination error.list_all/4paginates by hand instead — see the comment there for why a truncated list is the most dangerous value this module can return.Kubereq.PodExec.open?/1is broken:Kubereq.Connect.handle_call(:open?, …)returns a malformed two-tuple GenServer reply and crashes the connection process. Nothing here calls it; liveness is a monitor's job.Kubereq.PodExec.start_link/1raisesMatchErrorrather than returning{:error, _}when the websocket upgrade fails, and it links to whoever starts it and stops with the transport error as its exit reason.open_exec/5contains both: it starts the channel from a dedicated trapping owner, so neither the raise nor a later blip can reach the caller, and callers get{:exec_down, pid, reason}instead of an exit signal.
Non-2xx is {:ok, _}
kubereq installs only a request step (Req.Request.prepend_request_steps),
so nothing converts HTTP status into an error. A 404 arrives as
{:ok, %Req.Response{status: 404, body: %{"kind" => "Status", …}}}. Every
{:error, {:k8s, _}} in this backend is produced by normalize/1 below —
this module is where the vocabulary is created, not glue around one that
already exists. {:error, _} out of kubereq itself means a transport
failure or a %Kubereq.Error.StepError{}.
Summary
Functions
A Req.Request with kubereq attached, pointed at resource.
Send a close frame to a PodExec started by open_exec/5.
The API server URL of the current context — the enforcement-cache key.
POST a NetworkPolicy manifest.
POST a Pod manifest.
DELETE a NetworkPolicy by name.
DELETE a Pod with no grace period.
A client/2 for the exec subresource, negotiating v4.channel.k8s.io.
Run command to completion and return everything it wrote to stdout.
Run command and feed payload to its stdin over the exec websocket.
GET a NetworkPolicy by name.
GET a Pod by name.
The loaded kubeconfig for config.
Every Pod matching label_selectors, following continue to the last page.
The container's logs, as a single binary.
The namespace to operate in.
Start a long-lived Kubereq.PodExec delivering frames to into.
Poll/watch a Pod until callback returns true, or :timeout ms elapse.
Types
@type config() :: keyword()
Cluster connection config.
:kubeconfig, :namespace, :timeout, :exec_timeout, and the :req_adapter
test seam.
Functions
@spec client( config(), keyword() ) :: Req.Request.t()
A Req.Request with kubereq attached, pointed at resource.
Kubereq.attach/2 loads the kubeconfig eagerly — file reads, and possibly
an exec auth plugin subprocess — every time it is called with a pipeline
module. Every call in this module goes through here, and the reaper calls in a
loop, so the loaded %Kubereq.Kubeconfig{} is cached in :persistent_term
and the struct is what gets passed to attach/2.
@spec close_exec(pid() | nil) :: :ok
Send a close frame to a PodExec started by open_exec/5.
Never Kubereq.PodExec.open?/1 first: in 0.4.4
Kubereq.Connect.handle_call(:open?, _, _) returns a malformed two-tuple
GenServer reply, which crashes the very process being probed.
The API server URL of the current context — the enforcement-cache key.
POST a NetworkPolicy manifest.
POST a Pod manifest.
DELETE a NetworkPolicy by name.
DELETE a Pod with no grace period.
kubereq has no DeleteOptions option, so gracePeriodSeconds goes through
as a plain Req param. Zero because the sandbox holds no state worth draining
and a lingering terminating Pod is still a billed Pod.
@spec exec_client(config()) :: Req.Request.t()
A client/2 for the exec subresource, negotiating v4.channel.k8s.io.
Requesting the subprotocol is what makes exec exit codes available at all.
Without this header the API server falls back to v1 channel.k8s.io, whose
channel 3 carries a human string produced by the container runtime —
measured: "command terminated with non-zero exit code: Error executing in Docker Container: 7" under this cluster's runtime, but
"command terminated with exit code 7" under containerd. Parsing that is
parsing a runtime's prose.
Under v4 the same channel carries a JSON Status object instead, so the exit
code is a field. Measured against v1.35.6+orb1: exit 7 yields
%{"status" => "Failure", "reason" => "NonZeroExitCode", "details" => %{"causes" => [%{"reason" => "ExitCode", "message" => "7"}]}},
and a clean exit yields %{"metadata" => %{}, "status" => "Success"} — note
that success also produces a frame, which is what makes "no news is good news"
the wrong reading of channel 3.
kubereq never sets this itself, but Kubereq.Connect.connect/1 passes
req.headers straight into Mint.WebSocket.upgrade/4, so a header put here
reaches the wire.
@spec exec_once(config(), String.t(), [String.t()], keyword()) :: {:ok, binary()} | {:error, term()}
Run command to completion and return everything it wrote to stdout.
Bounded by :exec_timeout (default 15s). Kubereq.exec/4 blocks on a stream
whose only deadline is the 10s HTTP-101 upgrade; after the upgrade its
Mint.WebSocket.recv/3 waits :infinity. write/2 runs inside the session's
own call path, so an unbounded exec here would wedge the session forever.
Exec exit codes are not available: kubereq never negotiates
v4.channel.k8s.io, so channel 3 arrives as an undecoded {:error, binary}.
This is parity with Docker, whose detached exec also never reports status —
not a regression, but it does mean a successful return proves the command was
started, not that it succeeded.
Run command and feed payload to its stdin over the exec websocket.
This is the secret channel. The Kubernetes exec API has no env parameter
— pods/exec has no such field and kubectl exec has no --env — so
Docker's first-class Env array has no counterpart. The three ways to get a
provider key into a sandbox and why only one survives:
envin the Pod spec: puts the key in the Pod object, i.e. in etcd, readable by anyone withget pods. Trades an in-sandboxpsleak for a cluster-wide one.Secret+envFrom: same etcd residency, plussecretsRBAC and a second object to leak on crash.- stdin: the bytes travel on websocket channel 0. They never enter argv,
never enter the API object, and never appear in
kubectl describe.
opts takes :container, and passing it is not optional in practice: every
other exec in this module pins the container, this one did not, and on a Pod
with more than one container the API server picks. The env file is the secret
channel, so "whichever container the server picked" is the wrong place for it.
The sandbox Pod happens to have one container plus an already-exited init
container, so the omission worked by luck rather than by construction.
command must terminate on its own
It must not rely on stdin EOF — no bare cat > file. Closing the websocket
to signal EOF makes the API server tear the exec down before it writes the
channel-3 Status, so the command's exit code never arrives and every failure
reads as success. Measured on v1.35.6+orb1, same Pod, cat > /no-such-dir/x:
with a client-side close: [:connected, {:close, 1000, ""}] — no channel 3
without a client-side close: [:connected, {:error, "…ExitCode…1"}, …] — channel 3 presentSo bound the read instead. head -c <byte_size(payload)> > file consumes
exactly the payload and exits, the server sends the status, and the close frame
arrives on its own. This is why the caller passes a byte count rather than
letting the shell read to EOF.
Bounded by :exec_timeout like exec_once/4.
GET a NetworkPolicy by name.
GET a Pod by name.
@spec kubeconfig(config()) :: Kubereq.Kubeconfig.t()
The loaded kubeconfig for config.
Accepts a %Kubereq.Kubeconfig{} struct, a pipeline module, or a
{module, opts} tuple under :kubeconfig; defaults to
Kubereq.Kubeconfig.Default, which covers both a developer's ~/.kube/config
and an in-cluster ServiceAccount with no caller input.
Every Pod matching label_selectors, following continue to the last page.
Never Kubereq.list/3, and never its into: :stream form. Both are
wrong here, in the same silent direction:
- the plain form returns one page and no indication that there were more;
do_list_into_stream/4doesKeyword.put_new(params, :limit, 10)and its stream{:halt, :ok}s on a mid-pagination error, so a partial list is indistinguishable from a complete one.
A short list is not a cosmetic bug. CrowdControl.Reaper reads this as the
evidence of what is live: under the reconciliation table, a live sandbox
missing from this list is live? = no, stored? = yes, and the reaper deletes
the store record of a running, billed sandbox — orphaning it permanently.
Truncation must therefore be impossible, and any page failure must surface as
{:error, _} rather than a shorter list.
The container's logs, as a single binary.
This is the diagnostic channel the backend had none of. A sandbox that dies
during provisioning previously produced {:k8s, {:pod_not_ready, "CrashLoopBackOff"}} and nothing else — the operator's next step was
kubectl logs by hand, which is only possible if the Pod still exists, and
destroy/1 has usually removed it by then.
Bounded by construction, because a log fetch is a diagnostic and must never become the reason a teardown hangs:
follow: false— always, never overridable.Kubereq.logs/4's own docs sayfollow: true"keeps the connection alive which blocks the current process"; that isKubereq.PodLogs' job, not this one.tailLines(default 50) andlimitBytes(default 65536) so a chatty container cannot return a megabyte into a crash report.:previousfor the case that matters most — a container that already restarted, whose current logs are empty precisely because the interesting run is the previous one.
Built on Kubereq.PodLogs, not Kubereq.logs/4. The latter looks like the
obvious call and is a trap three ways: its body is a lazy Stream, so a
rejected upgrade raises at Enum time rather than at the call and escapes the
guard around it; enumerating it can raise WithClauseError from
Kubereq.Connect.create_stream/4, whose message inspects a %Mint.HTTP1{};
and its :follow defaults to true, so the obvious call blocks forever.
Returns {:ok, ""} when a container has genuinely produced nothing, so a
caller can tell "nothing to say" from "could not ask". A Pod whose container
never started answers 400 rather than an empty body — there is nothing to
read — so CrowdControl.Backend.Kubernetes falls back to the Pod's own
state.waiting.message for that case.
The namespace to operate in.
:namespace, else the current kubeconfig context's own namespace, else
"default". Requiring the option would be friction on the dev path for no
safety gain: the context namespace is what kubectl would use, and in-cluster
Kubereq.Kubeconfig.ServiceAccount populates it from the projected
namespace file. Both topologies land on the right answer unasked.
@spec open_exec(config(), String.t(), [String.t()], pid(), keyword()) :: {:ok, pid()} | {:error, term()}
Start a long-lived Kubereq.PodExec delivering frames to into.
The channel is not linked to the caller. Two kubereq 0.4.4 hazards make
that the only safe contract, and both are handled here and nowhere else:
- On a failed websocket upgrade Kubereq.Connect's
init/1raises aWithClauseError,GenServer.start_link/3returns{:error, _}, and Kubereq.Connect's own{:ok, pid} = …turns that into aMatchErrorin the starting process. Arescueonly reaches that if the starter traps exits: otherwiseproc_lib'ssync_waitnever converts the child's abnormal exit into a value and the link signal kills the starter outright, before any rescue can run. (Written unlinked on purpose: that module is@moduledoc false, so an autolink to it is a broken doc reference.) - The returned process stops with the transport error as its exit reason, so a routine websocket blip would kill a linked caller mid-session.
So the channel is started by a dedicated owner process, spawned unlinked,
which traps exits and holds the only link. The caller gets a plain value back
and cannot be killed by either hazard. When the channel dies the owner sends
into a {:exec_down, pid, reason} message, which carries the same
information the old {:EXIT, pid, reason} did — a consumer that wants to know
still learns immediately rather than at its next poll.
The owner monitors into and closes the channel if it dies, so the channel
cannot outlive its consumer. It is deliberately not supervised: an exec channel
has no meaningful restart, and a supervisor would keep one alive after its
reader was gone.
@spec wait_until( config(), String.t(), (map() | :deleted -> boolean() | {:error, term()}), timeout() ) :: :ok | {:error, term()}
Poll/watch a Pod until callback returns true, or :timeout ms elapse.
Kubereq.wait_until/5's :timeout is a Req receive_timeout on the watch,
not a wall-clock deadline — the caller supplies that.