Rewrites the ciphertext columns a plan names, against live traffic.
ADR-0002. run/2 takes a plan module - the one
Encryptor.Ecto.Migration compiled - and an option list whose :mode is
required, and returns a Encryptor.Ecto.Migrator.Report.t/0 on both arms:
MyApp.Encryption.CloakMigration
|> Encryptor.Ecto.Migrator.run(mode: :dry_run)
MyApp.Encryption.CloakMigration
|> Encryptor.Ecto.Migrator.run(mode: :write, resume: true)verify/2 is the read-only half (decision 10): the same plan, the same
classification, no writes, and a non-zero arm for any row that is not
already in the target state.
MyApp.Encryption.CloakMigration
|> Encryptor.Ecto.Migrator.verify(sample: :all)The library function is the interface and the mix encryptor.ecto.migrate
and mix encryptor.ecto.verify tasks are thin argument parsers over it,
because a production host runs releases and a release has no Mix: a rotation
reachable only from a developer's laptop against production credentials is
the opposite of the control it is supposed to be (decision 1).
What one pass does
For each field of each rewrite, in the order the plan declares them, and one
{schema, field, prefix} at a time:
- rows are visited in primary-key order with keyset pagination - never
OFFSET, which degrades quadratically and skips rows when the set shifts under it (decision 6); - every row is probed before it is rewritten, so the pass is idempotent by construction and the checkpoint is a performance record rather than a correctness one (decision 5);
- every write is a compare-and-swap against the exact bytes the migrator read, so a row the application wrote in the meantime is counted rather than clobbered with a re-encryption of stale plaintext (decision 4);
- every batch is one transaction, and the checkpoint row is written inside it (decision 6).
Encryptor.Ecto.Migrator.Pass holds the per-row order and the reasoning
behind it; this module decides what is visited and with what.
There is no default mode
Exactly one of mode: :dry_run or mode: :write (decision 7). A missing
mode is an ArgumentError, not a dry run: making dry-run the default trains
operators to add a flag they stop reading, and making write the default puts
an irreversible pass one typo away.
A dry run performs every read, every probe, every decrypt and every encrypt and discards the write, so it is an exact rehearsal of the work - including which rows fail to decrypt, how long it takes, and whether the checkpoint table is there.
mode: :write is refused outright - before a row is visited - while any
field in scope declares source_authenticated: false without a validate:
(ADR-0004 decision 3a). That field's legacy cipher cannot fail a decrypt on
wrong bytes, so nothing but the host's own check stands between a corrupt
row and a permanent, authenticated re-encryption of whatever it decrypted
to. A dry run and a verification are unaffected: they answer the question
the operator is supposed to ask first, and their counts say
:migratable_unverified while they do it.
Options
| Option | Default | |
|---|---|---|
:mode | required | :dry_run or :write |
:batch_size | 500 | Rows per transaction |
:resume | false | Start after the recorded cursor |
:prefix | nil | The schema prefix to visit; the repo's default when absent |
:checkpoint | :table | :none runs with no checkpoint at all |
:checkpoint_table | "encryptor_ecto_migration_checkpoints" | |
:on_error | :halt | :continue records the failure and finishes |
:only_scopes | nil | Visit only these scopes |
:except_scopes | [] | Visit every scope but these |
:only | nil | [{Schema, [:field]}], to narrow the plan |
:writing_key | nil | The wrapping key name a rotation's rows must claim |
:progress | no-op | Called with the report after each batch |
:prefix is singular and the plan carries none, per ADR-0002 proposed
amendment 6: a prefix is a deployment-time placement decision rather than a
fact about the schema, and a host with several prefixes loops run/2 over
its own list. No mode enumerates prefixes and no database catalog is read to
find them.
checkpoint: :none with resume: true is an ArgumentError: resuming from
a checkpoint that was never written is a request with no meaning.
A rotation is a pass with writing_key: set
A single scope's data-key rotation is this same tool with from: and to:
naming one declaration under one vault - and, without this option, it
rewrites nothing. Assumptions A11 and A12 leave the outgoing key version
decryptable, so both probes answer "already in the target state" for a row
written under the previous version and the pass writes nothing at all.
:writing_key supplies the one fact that fixes it: the name of the wrapping
key the rows in scope are supposed to claim. A row whose header names any
other key is rewritten - under whatever version is current, which this
package never selects (A10) - and a row already claiming it is left alone.
The name is the message's own, as Encryptor.Message.describe/1 reports it
for each of the header's encrypted data keys, and the operator running the
plan is where it comes from: nothing keyless can ask a vault for "this
scope's current version", and the operator who minted the version knows its
name. It is a comparison target and never an authorization input. A stale
name is self-correcting rather than dangerous - every row is classified
migratable and rewritten, each rewrite encrypts under whatever version is
actually current, and a second pass under the right name reports a clean
scope.
There is no rotation mode. mode: is still exactly one of :dry_run or
:write (decision 7): a rotation is one of those two with the option set,
the dry run being the census and the write the rewrite. A rotation adds no
report class either. A row whose header names another key is :migratable,
because both halves of that class are literally true of it - the probe
failed and the from load succeeded.
One literal key name belongs to one key holder, so writing_key: against a
:scoped-profile vault requires only_scopes: naming exactly one scope,
and against a :single-profile vault - whose scope holds one key holder
already - it requires no scope filter and permits none. only: stays
orthogonal: a rotation narrowed to some of the plan's columns is a partial
rotation. A field whose target this package cannot read a header claim out
of takes the load attempt instead, and the load attempt cannot answer the
rotation question at all, so a writing_key: pass whose scope holds such a
field is refused rather than run with that field silently answering "already
in the target state" for every row.
A rotation is verified by a second mode: :dry_run pass carrying the same
writing_key:, which reports an empty migratable count over the whole
scope. It is not verified by verify/2: a verification takes the load
attempt by design, and by A12 the outgoing version loads.
Which failures are exceptions and which are reports
A run that cannot start raises: an unknown option, a missing mode, a schema whose primary key cannot be paged over, a scope filter against a rewrite that has no scope column, a missing checkpoint table. None of those is about rows, and none of them is improved by being handed back as an empty report.
A run that started reports. {:error, report} means the pass found rows an
operator has to decide about; the report says which, and carries everything
the pass did before it stopped (decision 11). There is no mode that skips a
row silently, and no arm that exits zero with failures recorded.
What lives elsewhere
Encryptor.Ecto.Migrator.Census renders the SQL half of decision 10 - the
queries a DBA runs against the database with no application and no key.
The mix task family is mix encryptor.ecto.migrate and
mix encryptor.ecto.verify. source_authenticated: and validate:
are declared in Encryptor.Ecto.Migration and applied in
Encryptor.Ecto.Migrator.Pass; this module carries only the half that has
to be decided before any row is read, which is the --mode write refusal
below.
Summary
Types
The mode a run performs. There is no default (decision 7).
Every mode a pass can be executed in, verify/2's included.
How much of the scope a verification reads (decision 10).
Functions
Runs a plan, in exactly one of the two modes.
Classifies the plan's rows without writing anything (decision 10).
Types
@type mode() :: :dry_run | :write
The mode a run performs. There is no default (decision 7).
@type opts() :: [ mode: mode(), batch_size: pos_integer(), resume: boolean(), prefix: String.t() | nil, checkpoint: :table | :none, checkpoint_table: String.t(), on_error: :halt | :continue, only_scopes: [String.t()] | nil, except_scopes: [String.t()], only: [{module(), [atom()]}] | nil, writing_key: String.t() | nil, progress: (Encryptor.Ecto.Migrator.Report.t() -> any()) ]
@type pass_mode() :: mode() | :verify
Every mode a pass can be executed in, verify/2's included.
Separate from mode/0 because mode/0 is run/2's option, and
decision 7's "exactly one of two, with no default" is a statement about that
option. :verify is not a third thing an operator may ask run/2 for; it
is the mode verify/2 puts a pass in, and a mode: :verify reaching
run/2 is an unknown mode there exactly as :sideways would be.
@type sample() :: pos_integer() | :all
How much of the scope a verification reads (decision 10).
:all is the whole scope, visited with the same keyset pagination a run
uses. A positive integer is a random sample of that many rows per field.
Functions
@spec run(module(), opts()) :: {:ok, Encryptor.Ecto.Migrator.Report.t()} | {:error, Encryptor.Ecto.Migrator.Report.t()}
Runs a plan, in exactly one of the two modes.
Returns {:ok, report} when no row needed an operator's decision and
{:error, report} when one did. Raises before visiting anything when the
run cannot start - see "Which failures are exceptions and which are
reports".
@spec verify(module(), verify_opts()) :: {:ok, Encryptor.Ecto.Migrator.Report.t()} | {:error, Encryptor.Ecto.Migrator.Report.t()}
Classifies the plan's rows without writing anything (decision 10).
Returns {:ok, report} when every row it saw was :already_target or
:null, and {:error, report} otherwise. That is a stricter arm than
run/2's: a readable legacy row is a perfectly good dry run and a failed
verification, because the question here is whether the rotation is finished
rather than whether it can proceed.
MyApp.Encryption.CloakMigration
|> Encryptor.Ecto.Migrator.verify(sample: :all)Three jobs, one function. It is the acceptance test at the end of a rotation
(ADR-0004 decision 8, step 6); it is what a host runs on a schedule to
detect drift; and exiting zero over sample: :all is ADR-0004 decision 5's
primary signal that the mixed window has closed and that dropping legacy:
is due. The telemetry counter that decision names is a convenience beside
it - a counter at zero is evidence about traffic, and a cold partition
nobody reads reports zero while still holding legacy bytes.
Options
| Option | Default | |
|---|---|---|
:sample | :all | :all, or a positive integer of rows per field |
:prefix | nil | The schema prefix to visit; the repo's default when absent |
Deliberately no others. --prefix is here because a verification that
silently checked a different prefix than the pass wrote to would be worse
than no verification. The rest of run/2's options are absent because the
contract in ADR-0002's amended typespec has exactly these two, and a
verification whose scope can be narrowed in six ways is a verification whose
green is hard to read.
What it does per row, and what it does not
The probe and the classification are the pass's, unchanged
(Encryptor.Ecto.Migrator.Pass), which is the point: a verification that
reimplemented "is this row in the target state?" would be a second answer to
the question the migrator already answers, free to drift from it. What the
verify mode skips is the dump - see that module's "The third mode reads and
stops".
It takes the expensive half of the one probe. A rewrite reads an already-migrated row's header and skips it without a key (decision 5's short-circuit); a verification opens the bytes, because "it says it is ours" is what the keyless census answers and this is the function that exists to answer more than that. Same code, same classification, one mode flag - see the pass's "Two ways to probe".
A verification never halts on a row. It runs with on_error: :continue, so
a table with unreadable rows produces a count of them rather than a report
that stops at the first: an operator asking "is this finished?" is asking
about the whole scope, and a report that stopped at row one answers a
different question. It still exits non-zero - Report.verified?/1 is about
what was found, not about how the pass ended.
It writes no checkpoint and reads none. A verification is not resumable and has nothing to resume: it holds no lock, changes nothing, and can simply be run again.