Encryptor.Ecto.Migration behaviour (Encryptor.Ecto v0.7.2)

Copy Markdown View Source

The compile-time DSL for a migration plan (ADR-0002 decision 2).

A plan module names one repo and, per schema, the fields to rewrite and how the scope is resolved for its rows:

defmodule MyApp.Encryption.CloakMigration do
  use Encryptor.Ecto.Migration, repo: MyApp.Repo

  rewrite MyApp.Accounts.Customer do
    scope_from :account_id

    field :tax_id, from: MyApp.Cloak.Encrypted.Binary, to: MyApp.Encrypted.Binary
    field :notes, from: MyApp.Cloak.Encrypted.String, to: MyApp.Encrypted.String
  end

  rewrite MyApp.Reference.Code do
    scope :none
    field :value, from: MyApp.Cloak.Encrypted.Binary, to: MyApp.Encrypted.Binary
  end
end

The module compiles to a Encryptor.Ecto.Migrator.Plan.t/0, handed over by the __plan__/0 callback this DSL defines. Encryptor.Ecto.Migrator.run/2 and verify/2 take the plan module, so a host names a file it can read.

Why a plan is code

Everything in a plan is code already: the from and to sides are type modules, and which fields are encrypted is exactly the kind of fact that should be reviewed in a diff, versioned with the schema it describes, and deleted in a named commit when the migration is finished. Configuration would make the most consequential operation this package performs invisible to code review - a change to which columns get rewritten, against live production data, arriving without a diff. So there is no config path here, and adding one is not a small convenience: it is the property being refused.

The compile-time checks are the deliverable

ADR-0002 decision 2 promises that a plan which would fail on row one fails at mix compile. Row one is against live production data, from a shell, usually late in a change window, and a typo discovered there has already cost the rehearsal. So every check that can be made against the schema and the modules is made while the plan module compiles, and each failure is a CompileError naming the schema and field it came from:

  • the schema is a real Ecto.Schema, and every field and into: names a field on it;
  • scope_from names a real column, and a scope module resolves;
  • every from: module can read the bytes - resolve!/2 decides between Encryptor.Ecto.Migrator.Source, Ecto.ParameterizedType and Ecto.Type and refuses a module that is none of them;
  • every to: module can both load and dump at one consistent arity. The two sides are not symmetric even though the record states them in one clause: a from: module is the host's own legacy code and only has to load, while a to: module receives the rewritten bytes and must dump as well. A to: that cannot dump fails on row one just as loudly, and nothing else in this package would have caught it.

Code.ensure_compiled/1 precedes each of those, so a plan may name a module from its own application: the check waits for the parallel compiler rather than answering "not loaded" on a race it would lose intermittently.

from: and to: may be the same module

The migrator constructs both sides' params itself (ADR-0002 decision 3), so a plan may name one of this package's own types on both sides and nothing here rejects from: equal to to:: a data-key rotation under an unchanged declaration is that shape.

What the same-module spelling does not describe is an in-place declaration edit. A field moving from scope: :none to scope: :process, or gaining a :context pair, is a context change and therefore a full rewrite even though no type module changed - but both sides of a same-module spec read that module's current declaration, so the spec says a rewrite from the edited declaration into itself. It says nothing about the bytes already in the column, which were written under the form the module no longer has; and where the edit changed which key wrapped them - a scope strategy, a vault - the source side cannot read them at all.

An in-place declaration edit is migrated as two declarations (ADR-0002 decision 3, as amended 2026-09-13). Keep the old declaration as a module of its own - a second declaration of the same column, under the old params - and name it from:; the edited declaration is to:. The source side reads the old module's own declaration and the target side the new one, so the two params values differ because the two declarations do. That is the shape a cloak-to-encryptor plan's from: module already takes, so the migrator needs nothing new to run it and the field spec gains no source-side params or context option. The rewrite is pinned in test/encryptor/ecto/migrator_run_test.exs.

Adopting encryption on a plaintext column

into: names a different target column, which is the backfill leg of the expand/backfill/contract dance a never-encrypted column needs (ADR-0002 decision 8): plaintext lives in a text column, ciphertext must live in a binary one, and the DDL either side of the backfill is the host's own migration. Both columns are checked against the schema; which one is wider is not this package's business.

Folding a blind index into the rewrite

index: names a blind index column, and the pass then writes that index in the same update that writes the ciphertext, computed from the plaintext it already loaded for the rewrite (ADR-0004's Note of 2026-09-24, answering its open question Q1). Without it - the default - the rewrite writes ciphertext only, and the index is a separate backfill after the rewrite is verified (ADR-0004 decision 9), exactly as before.

rewrite MyApp.Accounts.Customer do
  scope_from :account_id

  field :contact_email,
    from: MyApp.Cloak.Encrypted.String,
    to: MyApp.Encrypted.String,
    source_authenticated: true,
    index: :contact_email_index
end

The column must be one a blind_index declaration on the schema names, over the encrypted field the pass writes - the field itself, or its into: column - and the plan fails at mix compile otherwise. The value is Encryptor.Ecto.BlindIndex.put_index/3's: the same declaration, the same normalization and the same key derivation, asked with :dump, under the row's own scope. A failure computing it is recorded against the row under a reason naming the index column, so it stays attributable to the index rather than to the rewrite.

It covers the rows the pass rewrites. A row already in the target state is skipped before anything is loaded, so its index is whatever the application wrote: put_index/3 has to be live in the host's changesets by the time the new type modules are, or the backfill still has rows to visit.

What the legacy cipher does not prove

A legacy stream cipher has no authentication tag, so a failed decrypt is not a signal: the bytes decrypt to something, and the migrator would re-encrypt that something into authenticated storage, permanently, and report it as a success (ADR-0004 decision 3). Two field options exist for that:

  • source_authenticated: false acknowledges it. The report then counts that field's rows :migratable_unverified rather than :migratable, in a dry run and in a verification alike, so no evidence claims verification that never happened; and the pass refuses to run in --mode write for that field unless validate: is declared with it.
  • validate: is a host-supplied (term() -> boolean()) applied to the loaded plaintext before it is re-encrypted (decision 3b) - a tax identifier is nine digits, a serialized map parses, a kept legacy hash column recomputes (decision 3c). A row it rejects is :undecryptable, handled like any other failure. There is no built-in generic validator: this package cannot know what a valid value looks like, and a printable?/UTF-8? check would be reassurance rather than a control.

Silence is allowed only where authentication is provable

ADR-0004's proposed amendment of 2026-08-28 answers Q2. A field whose from: is one of this package's own vault-backed types needs no declaration, unless that type also declares legacy: (below), because the package that wrote those bytes authenticates them and Encryptor.Ecto.Migrator.Source.vault_backed?/2 can prove it - that is the case above where from: names one of this package's declarations, whether it is the same module as to: or the earlier declaration a two-declaration edit keeps. Every other from: is the host's own legacy reader: a cloak cipher module, a legacy load/1, an unknown Source. This package's correctness obligation on that format is nil (decision 1) and it cannot tell an AEAD cipher from a stream cipher by looking, so it asks - at mix compile, where the question is cheap - and such a field must declare source_authenticated: explicitly, true or false. true is not a capability this package checks; it is the host asserting that someone looked at the legacy cipher, in a line a reviewer sees in the diff.

One of this package's own types that declares legacy: is not silent either (ADR-0004's Amendment of 2026-09-29). Its vault authenticates what the vault wrote, but a row the vault cannot read is read through the legacy: module, which is the host's legacy reader again - so a plan whose from: is such a type - a reverse plan run while legacy: is still set, above all - declares source_authenticated: exactly as the forward plan did.

A host upgrading across either change sees its plan stop compiling, with a message naming the field and saying what to write.

Summary

Types

One field's rewrite.

Where a compile-time failure came from, for the CompileError.

Callbacks

The compiled plan. Defined by use Encryptor.Ecto.Migration.

Functions

Declares a plan module. Takes repo:, and nothing else yet.

Declares one field to rewrite: from:, to:, and optionally into:, source_authenticated:, validate: and index:.

Opens a rewrite of one schema. Its body declares a scope strategy and one or more fields.

Declares the scope strategy: :none for a global field, or an Encryptor.Ecto.ScopeContext module.

Declares that the scope is read off the named column of each row.

Types

field_spec()

@type field_spec() :: [
  from: module(),
  to: module(),
  into: atom() | nil,
  source: Encryptor.Ecto.Migrator.Source.resolved(),
  source_authenticated: boolean(),
  validate: (term() -> boolean()) | nil,
  index: atom() | nil
]

One field's rewrite.

:from and :to are the modules the host writes. :into names a different target column for the backfill leg of an adoption migration, and is nil for the ordinary data-only case. :source is derived rather than written: it is what Encryptor.Ecto.Migrator.Source.resolve!/2 decided about :from while the plan compiled, kept here because ADR-0004 decision 2 fixes that resolution as a compile-time step and re-deriving it at run time would put the decision back where the record took it from.

:source_authenticated and :validate are ADR-0004 decision 3, and :source_authenticated is always present in the compiled spec even where the plan did not write it: an undeclared field is true only because the compile-time check proved it (see "What the legacy cipher does not prove"), so the engine reads one key rather than deciding provability again per row.

:index names the blind index column the pass writes beside the ciphertext (see "Folding a blind index into the rewrite"), and is nil for the default two-pass path.

meta()

@type meta() :: [file: String.t(), line: non_neg_integer()]

Where a compile-time failure came from, for the CompileError.

Callbacks

__plan__()

@callback __plan__() :: Encryptor.Ecto.Migrator.Plan.t()

The compiled plan. Defined by use Encryptor.Ecto.Migration.

Functions

__using__(opts)

(macro)

Declares a plan module. Takes repo:, and nothing else yet.

field(name, opts)

(macro)

Declares one field to rewrite: from:, to:, and optionally into:, source_authenticated:, validate: and index:.

source_authenticated: is required rather than optional wherever the from: type is not one of this package's own, or is one that declares legacy: - see "What the legacy cipher does not prove".

rewrite(schema, list)

(macro)

Opens a rewrite of one schema. Its body declares a scope strategy and one or more fields.

scope(strategy)

(macro)

Declares the scope strategy: :none for a global field, or an Encryptor.Ecto.ScopeContext module.

scope_from(column)

(macro)

Declares that the scope is read off the named column of each row.