Operations

Deploying is half the job. Once a release is running on Kubernetes you need to inspect it, attach to it, push the occasional hot-fix, and roll back when something does not behave as expected. Marea exposes a small, consistent set of commands for those day-2 tasks — every one of them, like the build/deploy commands, prints the underlying kubectl / helm invocation before it runs, so nothing is hidden.

This guide covers three workflows:

1. Attaching a remote IEx console to a running pod.
2. Pushing hot-fixes into a live release.
3. Rolling back a deploy.

All three assume you already have kubectl / helm configured and your --deploy / --release defaults set (Marea remembers the last ones used in <state_dir>/last_values).

Remote IEx console

marea k8s console --deploy <d> --release <r> [--pos <n>]

marea k8s console drops you into an IEx session attached to a running pod, in a second. There is no rebuild, no port-forward, no Erlang distribution wiring — Marea does the equivalent of:

kubectl exec -ti <pod> -n <namespace> [--context <ctx>] -- console

The console shell command is baked into the runtime image by the default Dockerfile template (priv/templates/docker/dockerfile_v01.eex). It is a one-line wrapper around <release>/bin/<release> remote, so your terminal becomes a true IEx remote shell against the live BEAM node — same prompt, same module reloading rules, full access to the running supervision tree. Type Ctrl-G q (or Ctrl-C a) to detach without taking the node down.

How the pod is selected:

• Pods are listed via kubectl get pods -l release=<r> -n <namespace>.
• --pos <n> (0-based) picks one directly. Without it Marea prompts interactively when more than one pod matches.
• The kube context comes from deploys.<d>.helm.kube_context if set, or your current kubectl context.

Because console is just a shell entry point inside the image, the same idea works without Marea: kubectl exec -ti <pod> console. The Marea command is the convenience wrapper that turns --deploy / --release into the right pod selector and namespace flags.

About kubectl exec and the deferred shell. iex needs to take over the user's terminal, so marea k8s console returns the final command via Marea's deferred shell mechanism. The escript exits, the shell wrapper executes the command, and the IEx prompt attaches directly to your terminal — no extra subprocess layer between you and the pod.

Hot-fixes

Most changes go through marea build helm --upgrade: rebuild the image, push it to ECR, update values.yaml, helm upgrade. That flow is reproducible, takes 1–2 minutes, and leaves a clean git trail.

Sometimes that is too slow — a tiny patch you want to verify against production traffic now, an investigation that needs a temporary instrumentation function in a running module, a fix you want to live on a single pod for a few minutes before committing to it. For those cases, Marea ships a hot-fix workflow:

marea remote --deploy <d> --release <r> --cookie <c> [--pos <n>]

marea remote opens a hidden Erlang node on your laptop and connects it to a single pod's BEAM via Erlang distribution. It is the moral equivalent of iex --remsh, with two pieces of glue:

1. The IEx session is started with --dot-iex pointing at priv/templates/k8s/remote.exs, which loads your locally compiled _build/dev/lib/*/ebin/ paths into the local node and connects to the remote node.
2. Helper functions in the Remote module let you push compiled .beam files into the remote node without rebuilding the image:

Remote.load(MyApp.Worker)        # hot-load on the connected pod
Remote.load_all(MyApp.Worker)    # hot-load on every node on that host
Remote.show(MyApp.Worker)        # compare local vs remote MD5
Remote.revert(MyApp.Worker)      # purge the patched code, reload from disk
Remote.revert_all(MyApp.Worker)  # revert across every node on that host

Under the hood, Remote.load/1 reads the local .beam and pushes it with :rpc.call(node, :code, :load_binary, [...]). There is no nodetool, no out-of-band agent — just standard OTP code-loading via distributed Erlang. Remote.revert/1 calls :code.purge/1 plus :code.get_object_code/1 on the remote so the next call to that module reloads the original beam from /opt/<release>/lib/....

When to reach for it

• ✅ A small patch you want to validate live before committing to a full image build.
• ✅ Adding :dbg traces or temporary IO.inspect/2 calls in a module to investigate a production-only bug.
• ✅ Restoring service after a deploy regression while you prepare a proper fix and rebuild.
• ❌ Long-lived changes. A pod restart, a horizontal pod autoscaler scaling event, or the next helm upgrade will wipe the patched module and pull the original beam from the image. Always follow up with marea build helm --upgrade.
• ❌ Anything involving struct definitions, defstruct, or callbacks changing a behaviour signature, unless you fully understand the hot-code-loading rules.
• ❌ Releases where the cookie has not been provisioned for outside attach. The pod and your laptop must share the Erlang cookie; in most setups that is a value in deploys.<d>.cookie or pulled from a secret.

Prerequisites

• Local code compiled (mix compile) — Remote.load/1 reads from _build/dev/lib/*/ebin/.
• Network reachability between your laptop and the pod's BEAM distribution port. A common setup is a kuttle-based tunnel that pipes Erlang distribution traffic over kubectl exec.
• Matching cookies and node names — Marea constructs the remote node name from the release name and pod IP and reads the cookie from --cookie or the deploy config.

Rollback

Two complementary paths cover the cases that come up in practice.

Git-tracked values.yaml: roll the source of truth back

Every marea build helm updates the deploy's <marea_dir>/deploys/<deploy>/values.yaml with the freshly built image reference and commits the change automatically (this is the last step of Marea.Plugins.Helm.store_image/1). That gives you a clean git history of every image rolled to that environment:

$ git log --oneline -- marea.d/deploys/staging/values.yaml
9f1b2c0 Updated marea.d/deploys/staging/values.yaml with my-org/api:2026_05_07-…
3d7a4ae Updated marea.d/deploys/staging/values.yaml with my-org/api:2026_05_06-…
b104e2f Updated marea.d/deploys/staging/values.yaml with my-org/api:2026_05_05-…

Rolling the cluster back to yesterday's image is then just two commands:

git checkout 3d7a4ae -- marea.d/deploys/staging/values.yaml
marea helm upgrade --deploy staging

This is the preferred rollback path when:

• You want git, the cluster, and values.yaml to stay aligned — a follow-up helm history shows the rolled-back version, and the next marea build helm builds on top of the same source of truth.
• Multiple values files (image refs, ConfigMap data, env settings) have moved together and you want to rewind all of them as a unit.
• You want to inspect or edit the values before re-applying them.

Commit the checkout (or revert the offending commit) so the rollback itself is recorded — the next marea build helm --upgrade will then land cleanly on top of it.

marea helm rollback: ask Helm to do it cluster-side

When you need to rewind right now and worry about git later:

marea helm rollback --deploy <d> --pos <n>

This is a thin wrapper over:

helm rollback <chart> <n> -n <namespace> [--kube-context <ctx>]

<n> is the revision number from helm history, which Marea also exposes:

marea helm history --deploy <d>

helm rollback is a destructive cluster-side operation, so Marea runs it through Lib.pause_cmd!/2 — the printed command pauses for an enter / Ctrl-C confirmation, unless you pass --nopause (e.g. in CI).

This path is the right one when:

• You need to revert in seconds and the source-of-truth alignment can wait.
• The values file in git already matches the version you want, but the cluster has drifted (e.g. someone ran helm upgrade manually).
• You are rolling back a Helm-only change (a templating fix, a resource limit tweak) where rebuilding the image would be wasted work.

After a helm rollback, the cluster's effective image does not match values.yaml in git anymore. Either follow up by checking out the matching values commit (so the two stay in sync) or accept that the next marea build helm --upgrade will roll forward off the git state, not the rolled-back state.

Choosing between them

In both cases the printed kubectl / helm command is the one you would run by hand — copy it out of the terminal if you ever need to debug a rollback that did not behave as expected.

Where to go next

• plugins/helm.md — full reference for helm chart, history, values, template, upgrade, delete, rollback.
• plugins/build.md — how build helm produces the git-tracked values.yaml that powers the rollback workflow.
• 05-architecture.md — deferred shell execution, the mechanism behind marea k8s console and marea remote.