How to migrate a host app off cloak_ecto

This guide takes a running multi-tenant application whose encrypted columns are written by cloak_ecto and leaves it with those same columns written by this package, with the application serving traffic throughout. It works unchanged for a host leaving a hand-rolled encrypted Ecto.Type: cloak is the named case because it is the common one, not because it is an integration.

It assumes you already know what changes when you make this move - per-scope keys, the encryption context, fail-closed scope, and what a blind index does and does not restore. If you do not, what changes when you move off cloak_ecto is that page, and it is worth reading before you run step 3.

The host in the examples is a payment application: a cards table with an encrypted pan and an encrypted notes, a signups table with an encrypted email and an A/B variant column, and one exact-match lookup - find a signup by email, served today by a Cloak.Ecto.SHA256 sibling column.

The steps below rewrite each encrypted column in place, which is the default and the shorter walk. A host that would rather keep the old column untouched until the very end can take the parallel-column exit instead: a new column beside each old one, backfilled and verified, then read.

Before you start

• The legacy key material still exists and the legacy modules still read the rows. A host whose cloak key is gone has rows that are already shredded. Nothing here can recover them, and the migration will classify every one of them :undecryptable.
• Vault key material exists for every scope you are about to migrate, or can be provisioned in step 1. Provisioning is encryptor's territory: this package creates and re-wraps no key, and none of its tasks touches one. Its one key-store delete is Encryptor.Ecto.KeyStore.shred/3, the end of a crypto-shred (ADR-0007, proposed), and nothing in this guide calls it.
• You can deploy twice, which is what steps 2 and 3 are, and you can run a command against production from your release.
• PostgreSQL. The census SQL at the end is written in Postgres; the tasks themselves are adapter-agnostic.

Every command below is given in two forms. The release eval form comes first and is the one to use: a rotation you can only run from a laptop holding production credentials is the opposite of a control. The mix form is the same call with an argument parser in front of it, for a host that runs migrations from a build machine.

eval starts nothing: not your repository, and not your vaults. Every eval below therefore goes through one function in your release module, which the release task sets up before step 4. The mix form needs none of it, because the task starts your application.

The runbook at a glance

The table says the point of no return is step 5, and in bulk it is. In practice reversibility ends at step 3: the moment the new type modules are live, ordinary application traffic starts writing new-format rows, and a revert leaves rows the old modules cannot read. Treat the step 3 deploy as the decision.

Step 0. Finish or abandon any in-flight legacy rotation

If your cloak vault is mid-rotation it has two ciphers configured, and both are readable through the same type module, so nothing here breaks. Two migrations at once make one report, which is the problem. Let the cloak rotation finish, or stop it, before continuing.

Nothing is written in this step and there is nothing to check but your own deploy state.

Step 1. Provision vault key material for every scope

Do this with encryptor's own provisioning, for every scope that has a row in a table you are about to touch - including scopes that are offboarded but whose rows are still present.

If it differs: a scope with no key material fails at the first row of its own, in step 4, with a vault error rather than a decrypt failure: the row is counted undecryptable, but its failure reason names Encryptor.Ecto.EncryptError where an unreadable legacy row says :load_failed. That is a provisioning gap, not a migration finding; go back and fill it.

Step 2. Deploy with both libraries in the tree

Add this package beside cloak_ecto and deploy. Schema fields still name the legacy type modules; nothing about how a row is read or written changes yet.

# mix.exs - both, for now. Step 8 removes the second.
{:encryptor_ecto, "~> 0.7.0"},
{:cloak_ecto, "~> 1.2"}

Expected: an ordinary deploy, with no behaviour change and no new writes in any new format.

If it differs: a compile failure here is a dependency conflict, and it is better found now than between steps 3 and 5, which is the whole reason this is a deploy of its own.

Step 3. Switch the type modules over, with legacy: set

The type modules change; the schemas do not.

defmodule MyApp.Encrypted.Binary do
  use Encryptor.Ecto.Binary,
    vault: MyApp.Vault,
    legacy: MyApp.Cloak.Encrypted.Binary
end

defmodule MyApp.Encrypted.String do
  use Encryptor.Ecto.String,
    vault: MyApp.Vault,
    legacy: MyApp.Cloak.Encrypted.String
end

legacy: makes reads tolerate both formats: the new load is attempted first, always, and the legacy module is tried only when the new one fails with a message-shaped failure. It is never a write path - a value read through the legacy module and written back is written in the new format, so ordinary traffic migrates rows on its own from this moment.

Wire the counter that tells you when the window has closed, in the same deploy:

:telemetry.attach(
  "legacy-load-counter",
  [:encryptor_ecto, :legacy_load],
  fn _event, %{count: n}, %{table: table, column: column}, _config ->
    MyApp.Metrics.increment("encryptor_ecto.legacy_load", n, table: table, column: column)
  end,
  nil
)

The event's metadata is the table and the column and nothing else - no value, no bytes, no reason, no scope.

Expected: the deploy goes out, reads keep working on every row, and the legacy_load counter climbs immediately. A counter that is flat at zero from the first minute means the types are not actually being exercised, or legacy: did not reach the module you thought it did.

If it differs: an exception naming a missing scope or a missing context is host misconfiguration and is deliberately loud - the legacy fallback is not attempted for either, because answering a configuration bug with a successful legacy read buys you a silent year of unmigrated rows. Fix the scope resolution and redeploy.

While this window is open, an unmigrated row is read under the legacy scheme's rules. For a cloak host that means no encryption context and no per-scope key separation for those rows. The guarantee is per-row until the pass finishes, which is why the window is meant to be short.

Before step 4: the release task

bin/my_app eval runs on a VM that has loaded your configuration and started nothing, which is also what a release migration task built on Ecto.Migrator.with_repo/2 gives you: the repository, and none of your supervision tree. Both vaults the pass reads through live in that tree - the legacy vault that legacy: and every from: decrypt with, and the vault the new type modules encrypt with - so neither is running.

Start them yourself, inside the callback, from the configuration the release has already loaded:

defmodule MyApp.Release do
  @app :my_app

  def with_encryption(fun) do
    Application.load(@app)

    {:ok, result, _apps} =
      Ecto.Migrator.with_repo(MyApp.Repo, fn _repo ->
        {:ok, _legacy} = MyApp.Cloak.Vault.start_link()
        {:ok, _vault} = MyApp.Vault.start_link()
        fun.()
      end)

    result
  end
end

Each start_link reads its keys exactly as it does when your application boots: the legacy vault reads its cipher and key from your configuration or its own init/1, and the new vault runs its init/1. Nothing here passes a key, and nothing should.

Expected: a step 4 dry run through this function reports the same counts as the mix form against the same rows.

If it differs: every row undecryptable, each failure reason a {:raised, SomeModule} naming what the legacy reader raised, is the legacy vault not running rather than a finding about your data. A cloak-shaped legacy vault keeps its ciphers in its own process, so a read before that process starts raises, and the migrator records the raise against the row. Start the vault and run it again before you investigate a single primary key. With the legacy vault up and only the new vault down, the reason on every row is {:raised, Encryptor.Ecto.EncryptError} instead, and the fix is the same.

Step 4. Rehearse with a dry run

Write the plan first. It is code, it is reviewed in a diff, and step 8 deletes it. Generate the skeleton if you want a starting point:

$ mix encryptor.ecto.gen.plan --module MyApp.Encryption.CloakMigration
* creating lib/my_app/encryption/cloak_migration.ex
7 candidate fields across 4 schemas. It does not compile yet: finish every `scope_from` and every `to:`, and delete the fields that are not encrypted. The file says which is which.

The generated file does not compile on purpose: every scope_from names :TODO_scope_column, every to: is a comment, and the field list over-reports. Its from: is the type module each field names today, which after step 3 is your new one; point it at the legacy module the rows were written with. Finish it into something like this:

defmodule MyApp.Encryption.CloakMigration do
  use Encryptor.Ecto.Migration, repo: MyApp.Repo

  rewrite MyApp.Payments.Card do
    scope_from :merchant_id

    field :pan,
      from: MyApp.Cloak.Encrypted.Binary,
      to: MyApp.Encrypted.Binary,
      source_authenticated: true

    field :notes,
      from: MyApp.Cloak.Encrypted.String,
      to: MyApp.Encrypted.String,
      source_authenticated: true
  end
end

source_authenticated: is required on every field whose from: is not one of this package's own vault-backed types, and the plan does not compile without it. Write true when you have checked that the legacy cipher authenticates - Cloak.Ciphers.AES.GCM does. Write false when it does not - Cloak.Ciphers.AES.CTR does not - and add a validate: beside it. On another rewrite in the same plan, a signups.email field on a CTR vault reads:

    field :email,
      from: MyApp.Cloak.Encrypted.String,
      to: MyApp.Encrypted.String,
      source_authenticated: false,
      validate: &MyApp.Encryption.Checks.email_matches_stored_hash?/1

If you have a legacy hash column beside the field, recomputing it over the loaded plaintext is the strongest validator available and it is free. It is also the reason step 7 drops that column after the rewrite rather than before. The sample output in the rest of this guide is from the two-field Card plan above, which is why its migratable_unverified count stays at zero.

Add the checkpoint table in the same change, with your own migration on your own deploy schedule:

$ mix encryptor.ecto.gen.migration
* creating priv/repo/migrations/20260828105920_create_encryptor_ecto_migration_checkpoints.exs

Then rehearse. A dry run performs every read, probe, decrypt and encrypt and discards the write, so it is an exact rehearsal, including how long it takes. Run it with on_error: :continue the first time: the default halts at the first unreadable row, and what you want from a rehearsal is the complete census of problems rather than the first one.

$ bin/my_app eval '
  {status, report} =
    MyApp.Release.with_encryption(fn ->
      Encryptor.Ecto.Migrator.run(MyApp.Encryption.CloakMigration,
        mode: :dry_run, on_error: :continue)
    end)

  IO.inspect(report.counts, label: "counts")
  IO.inspect(report.failures, label: "failures")
  if status == :error, do: System.halt(1)
'
counts: %{
  null: 412,
  undecryptable: 3,
  already_target: 1908,
  migratable: 2204551,
  migratable_unverified: 0
}
failures: [
  %{
    id: 88123,
    reason: :load_failed,
    field: :pan,
    schema: MyApp.Payments.Card
  },
  ...
]
$ echo $?
1

The same pass through mix:

$ mix encryptor.ecto.migrate MyApp.Encryption.CloakMigration --mode dry-run --on-error continue
mode: dry_run
null: 412
already_target: 1908
migratable: 2204551
migratable_unverified: 0
undecryptable: 3
concurrent: 0
failures: 3
  MyApp.Payments.Card.pan id=88123 reason=:load_failed
  MyApp.Payments.Card.pan id=88124 reason=:load_failed
  MyApp.Payments.Card.pan id=91002 reason=:load_failed
$ echo $?
1

Expected: exit 0 with undecryptable: 0, and already_target non-zero if step 3 has been live for a while - ordinary traffic has been migrating rows on its own. A failure line carries the schema, the field, the primary key and the reason, and never a value.

If it differs:

Step 5. Run the pass

This is the point of no return in bulk. The application keeps serving.

$ bin/my_app eval '
  {status, report} =
    MyApp.Release.with_encryption(fn ->
      Encryptor.Ecto.Migrator.run(MyApp.Encryption.CloakMigration,
        mode: :write, except_scopes: ["tnt_offboarded"])
    end)

  IO.inspect(report.counts, label: "counts")
  IO.inspect(report.failures, label: "failures")
  if status == :error, do: System.halt(1)
'

$ mix encryptor.ecto.migrate MyApp.Encryption.CloakMigration --mode write --except-scope tnt_offboarded
mode: write
null: 412
already_target: 1908
migratable: 2204551
migratable_unverified: 0
undecryptable: 0
concurrent: 37
failures: 0
$ echo $?
0

concurrent counts rows the application rewrote between the migrator's read and its write. They are not failures and not a class: every write is a compare-and-swap against the exact bytes that were read, so such a row was left alone rather than clobbered with a re-encryption of stale plaintext, and it was already counted once when it was read.

Expected: exit 0, failures: 0, and migratable equal to what the dry run predicted less whatever ordinary traffic migrated in the meantime.

If it differs:

Resuming reads the cursor recorded after each batch, inside that batch's own transaction:

$ mix encryptor.ecto.migrate MyApp.Encryption.CloakMigration --mode write --resume

--resume with --no-checkpoint is refused: resuming from a checkpoint that was never written is a request with no meaning.

Step 6. Verify over the whole scope

This is the acceptance test.

$ bin/my_app eval '
  {status, report} =
    MyApp.Release.with_encryption(fn ->
      Encryptor.Ecto.Migrator.verify(MyApp.Encryption.CloakMigration, sample: :all)
    end)

  IO.inspect(report.counts, label: "counts")
  if status == :error, do: System.halt(1)
'

$ mix encryptor.ecto.verify MyApp.Encryption.CloakMigration --sample all
mode: verify
null: 412
already_target: 2204551
migratable: 0
migratable_unverified: 0
undecryptable: 0
concurrent: 0
failures: 0
$ echo $?
0

Expected: exit 0, with every row already_target or null.

If it differs: verification is deliberately stricter than the pass. A table of readable legacy rows is a green dry run and a red verification, because "every row is in the target state" is the question the acceptance test has to ask. Exit 1 with a non-zero migratable means rows were missed - a scope you excluded, a prefix you did not visit, or a field you narrowed away with only:. Re-run step 5 for them. A verification never halts on a row, so a table with unreadable rows gives you a count rather than a report that stops at the first.

Keep running this on a schedule afterwards. Exit 0 over sample: :all is also the primary signal that the mixed window has closed and step 8 is due.

Step 7. Replace the legacy lookup column

Cloak's answer to exact-match lookup on an encrypted column is a deterministic sibling column, so a host that needed lookups arrives here with one. It is replaced, not supplemented: adding a keyed index beside an unkeyed one fixes nothing while the unkeyed one remains.

Start by finding out which of three cases you are in, because it decides whether this step is optional and what its last action is. Read the type module named on the sibling field in your cloak-era schema.

Which case you are in

This is ADR-0004 decision 9. The three are not variations on one disposition, and treating them as one is how the unkeyed column survives a migration whose whole purpose included removing it.

Cloak.Ecto.SHA256 is a fingerprint with no key in it. Anyone holding a dump - a backup, a replica, a stolen snapshot, a support export - recomputes sha256(value) for every candidate value they care to guess and matches it against the column, with no key material and no access to your application. The columns hosts most want to look up on are email addresses, phone numbers and tax identifiers, which are exactly the guessable spaces. This is the unkeyed folk pattern Encryptor.Ecto.BlindIndex's Security properties names as the thing a keyed index exists to replace: its second table row, "the dump, and a guess at a plaintext they think is present", is the property this column does not have. Encrypting the value column and leaving this one beside it means the ciphertext is doing nothing for any guessable value.

So the drop is unconditional. If you decide not to adopt a blind index at all, you still drop the column, and you lose that lookup - which is the honest price of a disclosure you were paying for it all along.

A keyed Cloak.Ecto.HMAC or Cloak.Ecto.PBKDF2 column is a real blind index, and a dump alone tells an attacker nothing about the values in it. What it lacks is the domain separation ADR-0003 decision 2 builds into the HKDF info string. It is derived from one key configured for the deployment, so:

• two scopes storing the same value produce identical bytes, which is exactly the derive: :global column of the Security properties table - equality structure across the whole table rather than within a scope;
• two columns holding the same value produce identical bytes, so a dump can be joined across every table configured with that key - a signups.email_hash against a contacts.email_hash - which a per-column info string makes structurally impossible;
• a staging database restored from a production backup keys the same way, since there is no per-deployment :derivation_salt under the construction;
• and it does not shred with a scope's key. Destroy a scope's vault key material and this column still answers "does this scope have a row with value X" for anyone holding the legacy key, which is the last row of that table and the reason derive: :global has to be written out loud.

Replacing it is therefore worth doing even though it is not the emergency the unkeyed case is. Lookups keep working through the legacy column while the new one backfills, so the switchover has no gap in it.

No sibling column means you are choosing, not repairing, and the choice has its own cost - a blind index is a leakage decision paid per column and permanently. Reserve one for high-cardinality exact-match keys. In the example host, signups.email earns an index and the A/B variant column does not: a two-value column publishes its own distribution to anyone counting rows per distinct index value, with no key at all. Encryptor.Ecto.BlindIndex's What this does not defend against is the full statement, and an index on a low-cardinality column is a defect there rather than a tradeoff.

The order, which is fixed

Both of the tempting alternatives are wrong in a way that is invisible at the time:

1. Add the new index column in your own migration, and declare it on the schema.

   import Encryptor.Ecto.BlindIndex
   
   schema "signups" do
     field :merchant_id, :string
     field :variant, :string
   
     field :email, MyApp.Encrypted.String
     field :email_index, :binary
     blind_index :email, :email_index, normalize: :email
   end

   The column is yours - this package writes no migrations and adds no fields. It is a nullable :binary, 32 bytes at the default :bits, and it is added beside the legacy one rather than over it:

   defmodule MyApp.Repo.Migrations.AddSignupEmailIndex do
     use Ecto.Migration
   
     def change do
       alter table(:signups) do
         add :email_index, :binary
       end
   
       create index(:signups, [:merchant_id, :email_index])
     end
   end

   On Postgres that is:

   ALTER TABLE "signups" ADD COLUMN "email_index" bytea;
   CREATE INDEX "signups_merchant_id_email_index_index"
       ON "signups" ("merchant_id", "email_index");

   The database index is on (merchant_id, email_index) rather than on the index column alone, because every lookup is already inside a scope.

   Leave the legacy column in place in this migration. It is still serving lookups in the keyed case, and it is still the validator in both.

2. Write the index from your changesets with Encryptor.Ecto.BlindIndex.put_index/3, and read it with where_eq/3. Both compute through one place, so a write and a read cannot disagree about normalization or width.

   def changeset(signup, attrs) do
     signup
     |> cast(attrs, [:email, :variant])
     |> put_index(:email, :email_index)
   end
   
   def by_email(email) do
     MyApp.Signup |> where_eq(:email, email) |> MyApp.Repo.one()
   end

   Keep writing the legacy column in this deploy too, from the same changeset. That is what makes the switchover gapless, and it is the state step 4 ends. On a truncated index (bits: 64, 128 or 192) the read helper is where_eq_candidates/3 instead and it returns a set you filter after decrypting - the name is the contract, so a call site cannot forget.

   Audit every other write path while you are here. Encryption lives in the type and cannot be forgotten; the index lives in the call site and can. A bulk insert, an admin script or a second changeset function that skips put_index/3 produces a row whose lookup silently misses.

3. Backfill the index column, batched and under its scope, after the ciphertext rewrite of step 5 is verified. Two backfills at once make one report, and a failure in either stops being attributable to one of them.

   The backfill is a decrypt-and-recompute pass over your own rows, so it needs the scope the index derivation reads. There is no way around the decrypt: the index is computed from plaintext and nothing in the stored ciphertext can be transformed into one.

4. Switch reads over, and only then drop the legacy column - in the same migration that stops writing it, and not before.

   Not before, because while step 4's validate: was recomputing it, it was the best integrity check available on an unauthenticated source. Not later, because a column nobody writes but nothing dropped is a column that goes stale silently while still disclosing everything it disclosed before.

   defmodule MyApp.Repo.Migrations.DropSignupEmailHash do
     use Ecto.Migration
   
     def change do
       alter table(:signups) do
         remove :email_hash, :binary
       end
     end
   end

   ALTER TABLE "signups" DROP COLUMN "email_hash";

   Before you run it, confirm the new column is fully backfilled. This is SQL over your own tables, it needs no key, and it is the check that catches a backfill that skipped a scope:

   -- rows with a value but no index: must be zero before the drop
   SELECT count(*) AS unindexed
   FROM "signups"
   WHERE "email" IS NOT NULL
     AND "email_index" IS NULL;
   
   -- and per scope, which is where a partial backfill actually shows up
   SELECT "merchant_id",
          count(*) FILTER (WHERE "email_index" IS NOT NULL) AS indexed,
          count(*) FILTER (WHERE "email" IS NOT NULL) AS encrypted
   FROM "signups"
   GROUP BY 1
   HAVING count(*) FILTER (WHERE "email_index" IS NOT NULL)
        < count(*) FILTER (WHERE "email" IS NOT NULL);

   The second query returning no rows is the precondition. A scope listed there is a scope whose lookups break the moment the legacy column is gone.

   In the unkeyed case there is nothing to preserve and no reason to wait past step 6's exit 0: if you are not adopting an index, this migration is the whole of step 7. In the keyed case, the legacy key outlives the column unless you remove it - it is cloak configuration, and step 8 is where it goes with the rest of the legacy vault.

Expected: lookups return the same rows they did through the legacy column, for values that normalize the same way.

If it differs: a lookup that finds nothing is almost always normalization - the legacy hash and the new index were computed over differently normalized inputs. Normalization is lossy and directional, and changing it invalidates every value already stored in the column.

Four facts about this column decide when you can do it, rather than how:

• Rotating the vault's :derivation_salt invalidates every blind index in the deployment: it is a full reindex, not a rekey.
• Bumping an index's :version, changing its :normalize, or changing its :bits invalidates that column for the same reason. Each is a reindex.
• Rotating a scope's key requires reindexing that scope's rows as well. The derivation consults the current key material, so that scope's index keys change without any declaration changing, and lookups for that scope miss until the reindex lands. Encryptor.Ecto.BlindIndex's own documentation is the full statement.
• Turning :slow (Argon2id before the HMAC) on or off invalidates that column too, for the same reason: it changes what the HMAC is taken over. A column written under a slow: true declaration before the option was wired holds plain-HMAC bytes and is invalidated by the wiring itself, which is the one reindex on this list that no change of yours triggers.

A dump taken before the drop still contains the old column. That is a fact about your backups, and this package cannot fix it. It is worth saying plainly in the unkeyed case: every backup taken while email_hash existed still discloses every guessable email address in it, to whoever can read that backup, forever. Dropping the column stops the disclosure growing; it does not undo it. Whether that means re-taking your backup set, shortening its retention, or accepting it, is your call and your retention policy's, not this package's.

The full leakage table - what an attacker learns from a derive: :per_scope index and from a derive: :global one, holding a dump, a guess, one scope's key, or a retained dump after a shred - is Encryptor.Ecto.BlindIndex's Security properties, at the declaration where a reviewer meets it. The records behind this step are ADR-0004 decision 9 for the three dispositions and the ordering rule, and ADR-0003 for the construction.

Step 8. Close the window in one named commit

The precondition is evidence, not memory: step 6 exits 0 over the whole scope, and the [:encryptor_ecto, :legacy_load] counter has read zero for a full retention period. A quiet hour is not enough - a table with a cold partition nobody reads reports zero while still holding legacy rows.

The window is per-field, and so is its end. A host with twelve encrypted columns finishes eleven and still has one legacy reader open; the counter's metadata names the table and the column so the last one is identifiable.

In one commit:

• drop legacy: from every type module,
• drop cloak_ecto from mix.exs,
• delete the plan module,
• delete the legacy type modules and their vault, once nothing names them.

defmodule MyApp.Encrypted.Binary do
  use Encryptor.Ecto.Binary, vault: MyApp.Vault
end

Expected: the suite is green, the deploy is ordinary, and the legacy_load counter stays at zero because there is no longer an event to emit.

If it differs: a decrypt failure after this deploy is a row that was never migrated, and it is now unreadable. Revert the commit - which is why this step is last, and why the legacy key material and modules had to survive until now.

If you must go back: the reverse plan

There is no rollback tooling, deliberately: an automated rollback of a security migration is a mechanism whose only rehearsal is the emergency it exists for. What there is is a symmetry. The migrator constructs both sides' parameters itself, so a plan with from: and to: swapped is a valid plan, and running it walks the table back.

defmodule MyApp.Encryption.CloakRollback do
  use Encryptor.Ecto.Migration, repo: MyApp.Repo

  rewrite MyApp.Payments.Card do
    scope_from :merchant_id

    field :pan,
      from: MyApp.Encrypted.Binary,
      to: MyApp.Cloak.Encrypted.Binary,
      source_authenticated: true
  end
end

Its preconditions, all of which step 8 destroys:

• the legacy key material still exists,
• the legacy type modules are still in the tree,
• cloak_ecto is still a dependency.

Rehearse it with mode: :dry_run exactly as you rehearsed the forward pass. Each field declares source_authenticated: with the same answer the forward plan gave it. Its from: is one of this package's own types, but that type still declares legacy: until step 8, so a row the pass has not rewritten is read through the cloak module - and the plan does not compile without the declaration.

The other exit: a parallel column

The in-place walk above is the default. This is the other shape: a new binary column beside each encrypted one, named <field>_encrypted by convention - pan_encrypted beside pan - which the host's schema writes alongside the old one during the window. The migrator backfills it, reads stay on the old column until the backfill verifies, one deploy moves reads to the new column, and only then do the old column and the legacy library go.

What it buys is an old column nobody but the application writes: the pass reads it and never writes it, so until the drop the way back from any step is reverting a deploy. What it costs is a second column per field for the length of the window, a dual write in the host's own changeset, and one more deploy.

The package ships no DDL and no dual-write type for this. The two columns are the host's own migrations, and the dual write is the host's own changeset. The package's part is the backfill leg, which is the same into: a plan uses to adopt encryption on a plaintext column.

Parallel step 0. Finish or abandon any in-flight legacy rotation

The same as step 0 of the in-place walk, with one thing to know about checking it. The census a parallel plan renders reads the new column, because that is the column the pass writes, so before step 5 its format query shows only the rows the dual write has written and never a legacy one. To see which formats the old column holds, run the same format query over the old column yourself:

SELECT substring("pan" from 1 for 4) AS header,
       count(*) AS rows
FROM "cards"
WHERE "pan" IS NOT NULL
GROUP BY 1
ORDER BY 2 DESC;

Expected: the prefixes of the legacy format and nothing else. Two legacy prefixes growing against each other is a legacy rotation still running.

Parallel step 1. Provision vault key material for every scope

The same as step 1 of the in-place walk.

Parallel step 2. Deploy with both libraries and the new columns

Add this package beside cloak_ecto, as step 2 of the in-place walk does, and add the new columns in your own migration. They are nullable binary columns: a row the backfill has not reached is NULL there.

alter table(:cards) do
  add :pan_encrypted, :binary
  add :notes_encrypted, :binary
end

Expected: an ordinary deploy, with every new column NULL on every row.

Parallel step 3. Deploy the dual write

The old fields keep their legacy types and stay the fields the application reads. Beside each, the schema declares its <field>_encrypted pair on this package's type with no legacy:, because nothing but the new format is ever written into it, and the changeset writes both.

defmodule MyApp.Encrypted.Binary do
  use Encryptor.Ecto.Binary, vault: MyApp.Vault
end

defmodule MyApp.Encrypted.String do
  use Encryptor.Ecto.String, vault: MyApp.Vault
end

schema "cards" do
  field :merchant_id, :string
  field :pan, MyApp.Cloak.Encrypted.Binary
  field :notes, MyApp.Cloak.Encrypted.String
  field :pan_encrypted, MyApp.Encrypted.Binary
  field :notes_encrypted, MyApp.Encrypted.String
end

@pairs [pan: :pan_encrypted, notes: :notes_encrypted]

def changeset(card, attrs) do
  card
  |> cast(attrs, [:merchant_id | Keyword.keys(@pairs)])
  |> dual_write()
end

defp dual_write(changeset) do
  Enum.reduce(@pairs, changeset, fn {old, new}, changeset ->
    case fetch_change(changeset, old) do
      {:ok, value} -> put_change(changeset, new, value)
      :error -> changeset
    end
  end)
end

The dual write has to be live before step 5, and it has to cover every write that changes the old column. The pass's compare-and-swap compares the new column only: a row the application writes while the pass holds it is left alone because its new column changed, and an application write that changed only the old column would leave the pass free to store a value that is already stale. A write that bypasses the changeset - an update_all, a raw SQL statement - is a write the dual write does not see, so find those before this deploy.

The encryption context of a new column binds its own name: this package's types derive the declared "column" from the field, so pan_encrypted's rows are written under "pan_encrypted". Step 8 comes back to that.

Expected: a new row carries both columns, the old one in the legacy format and the new one in this package's. An update carries both columns of every field it changes; a field it leaves alone keeps its pair as it was, NULL until step 5 reaches it. Reads are unchanged.

If it differs: a field an update changed whose new column is still NULL, or still holds the value from before the update, is a write path the dual write does not reach. Step 5 backfills a NULL pair but not a stale one: the pass and step 6 both find a stale pair already in the target state, so an uncovered write path is found here or not at all.

Parallel step 4. Write the plan and rehearse it

Each field reads the old column through the legacy type module the rows were written with and writes into: its <field>_encrypted pair through this package's type, the one step 3 declared without legacy::

defmodule MyApp.Encryption.CloakParallel do
  use Encryptor.Ecto.Migration, repo: MyApp.Repo

  rewrite MyApp.Payments.Card do
    scope_from :merchant_id

    field :pan,
      from: MyApp.Cloak.Encrypted.Binary,
      to: MyApp.Encrypted.Binary,
      into: :pan_encrypted,
      source_authenticated: true

    field :notes,
      from: MyApp.Cloak.Encrypted.String,
      to: MyApp.Encrypted.String,
      into: :notes_encrypted,
      source_authenticated: true
  end
end

source_authenticated: is required exactly as in step 4 of the in-place walk, with the same answers and the same validate: for a cipher that does not authenticate. The checkpoint table, the release task and the dry run are that step's too, with this plan's name in place of the in-place one.

Expected: exit 0 with undecryptable: 0. The rows the dual write has already written count already_target; every other row with a value counts migratable.

If it differs: the in-place step 4 table applies unchanged.

Parallel step 5. Write the new columns

Run the plan in mode: :write, with the same options and the same resume as step 5 of the in-place walk. The pass reads each old column and writes only its pair: every old column is byte for byte what it was before the pass, and a row whose old column is NULL leaves its pair NULL and counts null.

Expected: exit 0, failures: 0. The integrity query the plan's census renders compares the two columns: target_non_null reaching source_non_null is the backfill catching up, and a non-zero target_empty is the failure it watches for.

Parallel step 6. Verify over the new columns

$ mix encryptor.ecto.verify MyApp.Encryption.CloakParallel --sample all

verify/2 reads the column the plan writes, so over this plan it reads the <field>_encrypted columns. The release eval form is step 6's with this plan's name.

Expected: exit 0, with every row already_target or null.

If it differs: a non-zero migratable is a row whose new column the pass did not reach or a write path the dual write missed. Re-run step 5 and look for the write path.

Reads are still on the old columns here, and nothing has depended on the new ones yet. There is no legacy_load count in this shape - the new types carry no legacy: - so exit 0 over sample: :all is the whole of the signal that step 7 is due.

Parallel step 7. Cut reads over, in one deploy

Move every read of an encrypted field to its <field>_encrypted pair, in one deploy of its own. Keep the dual write in this deploy: the old columns stay current, so reverting it is the whole of the way back.

A legacy lookup column is dealt with as in step 7 of the in-place walk, with the keyed index declared over the <field>_encrypted field, since that is the field reads now come from.

Expected: every read succeeds, and every value read equals what the old column held.

If it differs: a DecryptError on a row step 6 verified is a read through a declaration whose "column" is not the one the pass wrote under - see step 8's note on renaming.

Parallel step 8. Drop the old columns and the legacy library

Once step 7 has been live long enough to trust:

• remove the old fields and the dual write from the schema and the changeset,
• delete the plan module, the legacy type modules and their vault,
• drop cloak_ecto from mix.exs,
• then drop the old columns in your own migration, once no deployed code names them.

A host that wants the old field names back can have them without re-encrypting anything, provided it pins the context to the name the rows were written under:

field :pan, MyApp.Encrypted.Binary, source: :pan_encrypted, column: "pan_encrypted"

Without column: the declared context would be derived from the field name, "pan", and every row would be refused. The same pin keeps a later physical rename of the column readable.

Expected: the suite is green and the deploy is ordinary.

Why there is no reverse plan here

The pass never writes an old column, and the dual write keeps every old column current until step 8. So up to step 7 the way back from any step is reverting a deploy: the application reads the old columns again and the new ones are ignored. After step 8 there is no old column to walk back into, which is also where the in-place walk's reverse plan stops being possible.

Watching a pass without a key

The census is SQL over your own tables. It connects to nothing, decrypts nothing, and is what you hand a DBA who should not be handed a key. Render it from the plan:

$ bin/my_app eval '
  MyApp.Encryption.CloakMigration
  |> Encryptor.Ecto.Migrator.Census.queries()
  |> Encryptor.Ecto.Migrator.Census.script()
  |> IO.puts()
'
-- "cards"."pan": format census, grouped on 4 bytes
SELECT substring("pan" from 1 for 4) AS header,
       count(*) AS rows
FROM "cards"
WHERE "pan" IS NOT NULL
GROUP BY 1
ORDER BY 2 DESC;

-- "cards"."pan": nothing became NULL or empty
SELECT count(*) AS rows,
       count("pan") AS non_null,
       count(*) FILTER (WHERE octet_length("pan") = 0) AS empty
FROM "cards";

-- "cards"."pan": rotation progress for one scope
SELECT count(*) FILTER (
         WHERE substring("pan" from 1 for 4) = :current_header
       ) AS done,
       count(*) FILTER (WHERE "pan" IS NOT NULL) AS total
FROM "cards"
WHERE "merchant_id" = :scope;

-- ... and the same three queries again for "cards"."notes", and for every
-- other field of every other rewrite the plan declares.

Substitute :scope with the scope you are watching, and :current_header with the prefix the format census shows growing - that is how you get it without a key.

The prefix is four bytes, and one byte would lie. A cloak envelope opens with a reserved 0x01 byte and this package's messages open with a version byte of their own, so a census keyed on the first byte alone can read as identical across both formats: one header, all migrated, over a table that is half legacy. Do not narrow it. The queries group rather than test, so a host whose formats collide further in gets a census that looks wrong rather than one that lies.

Run the integrity query before the pass and after it, and compare. Nothing should have become NULL and nothing should have become empty.

Related

• What changes when you move off cloak_ecto - the semantics behind every step here.
• mix help encryptor.ecto.migrate and mix help encryptor.ecto.verify - the full flag tables and exit codes, which live with the code that parses them.
• ADR-0004 - the record this runbook renders.