Fallback recovery

Fallback recovery lets offline trusted operators recover KEK or principal private-key material using registered fallback public keys.

This is break-glass. Keep fallback private keys offline and out of HTTP paths.

What fallback protects

KMS can store fallback envelopes for:

• KEK master keys;
• principal private-key material.

Fallback envelopes are encrypted to registered fallback public keys. Private keys are required to unwrap.

When fallback public keys are registered or configured, KEK creation and rotation commit only after every required fallback envelope is stored. If envelope encryption or persistence fails, KMS returns {:error, :fallback_creation_failed} and rolls back the KEK change. With no fallback public keys, KEK lifecycle operations remain unchanged.

Security rules

• Never send fallback private keys through ordinary HTTP clients, proxies, logs, or APM.
• The compiled HTTP router exposes no fallback operations, including public-key registry and envelope metadata calls.
• Run all fallback operations from trusted in-process Elixir code or a controlled remote shell on the KMS host.
• Store fallback private keys offline with separate custody.
• Test recovery before production.

Register a fallback public key

{:ok, public_key} =
  KMS.create_fallback_public_key(%{
    name: "ops-2026",
    curve: "X25519",
    public_key: public_key_material
  })

Lookup:

keys = KMS.list_fallback_public_keys()
{:ok, key} = KMS.fetch_fallback_public_key_by_name("ops-2026")

Bootstrap fallback keys

KMS can bootstrap configured public keys during startup:

config :kms, :fallbacks_bootstrap, true

config :kms, :fallback_public_keys, [
  %{name: "ops-2026", curve: "X25519", public_key: "..."}
]

Use this for deployment-managed public keys. Keep private keys outside normal deployment config.

KEK fallback recovery

List fallback envelopes for a KEK:

fallbacks = KMS.list_kek_fallbacks(kek.id)

Unwrap from trusted local code:

{:ok, master_key} = KMS.unwrap_kek(kek.id, "ops-2026", fallback_private_key)

Use recovered material only inside a controlled recovery runbook. Do not print it. Do not paste it into chat, logs, shell history, or tickets.

Principal fallback recovery

List principal fallback envelopes:

fallbacks = KMS.list_principal_fallbacks(principal.id)

Unwrap from trusted local code:

{:ok, private_key_payload} =
  KMS.unwrap_principal_private_key(principal.id, "ops-2026", fallback_private_key)

Rotate fallback public key

Use rotate_fallback_public_key/4 from trusted local code:

{:ok, rotated} =
  KMS.rotate_fallback_public_key(
    "ops-2026",
    %{name: "ops-2027", curve: "X25519", public_key: new_public_key},
    old_private_key
  )

Keep old private key until rotation and restore tests prove all required fallback envelopes are usable with the new key.

Incident runbook shape

1. Freeze changes to affected KEKs/principals.
2. Verify operator identity and break-glass approval.
3. Run unwrap from trusted KMS host/in-process shell.
4. Use recovered material for the narrow recovery objective.
5. Rotate exposed sessions/secrets after recovery.
6. Record audit evidence without secret payloads.
7. Rehearse restore from backups after incident close.

Failure modes

• Missing fallback private key: fallback envelope cannot be used.
• Wrong public-key ref/name: unwrap fails.
• Deleted fallback rows: recovery path gone.
• Lost RMK and no fallback: persisted KMS secrets may be unrecoverable.
• Logged fallback private key or unwrapped material: treat as secret spill.

Fallback recovery is not a substitute for database/RMK backups. It is an additional break-glass path.