An encrypted :binary field, declared the way cloak_ecto declares one.
A host writes one module per encrypted type and names it from the schema:
defmodule Payments.Encrypted.Binary do
use Encryptor.Ecto.Binary, vault: Payments.Vault
end
defmodule Payments.Cards.Card do
use Ecto.Schema
schema "cards" do
field :merchant_id, :string
field :pan, Payments.Encrypted.Binary
end
endThe generated module implements Ecto.ParameterizedType, so changesets,
queries and Repo calls keep their ordinary form and the encryption happens
in the type rather than at the call site (ADR-0001 decision 1).
The option set is closed
| Option | Required | Meaning |
|---|---|---|
:vault | yes | The Encryptor.Vault module this type encrypts through |
:tenant | no | :scope (default), :none, or a module implementing Encryptor.Ecto.TenantContext (see "A global field" below) |
:context | no | Static extra context pairs merged into every operation |
:legacy | no | The migration window's legacy type: load through it when the primary load fails (see "The migration window" below) |
:table, :column | no | Overrides for the frozen declared context values, writable at the use or at the field |
An unknown option raises while the host module compiles, and there is no
Application environment fallback for :vault: the vault is named at the
declaration or the module does not compile (decision 3). Every option that
could weaken the encryption context is one nobody gets, which is what makes
the closed set worth its inconvenience.
What goes in the encryption context
"table" and "column", derived once at declaration time from the schema's
source and the field's name, plus whatever :context adds (decision 4). The
tenant is not a context pair: it passes to the vault as key:, and the
vault derives and injects "tenant_ref" itself (acceptance amendment 1). A
declared "table" or "column" wins over a :context pair of the same
name - the derived values are the anti-substitution property and a static
pair cannot be allowed to shadow them.
Schema prefixes are deliberately absent: a prefix is a deployment-time placement decision, and binding it would make a ciphertext un-restorable into a differently-prefixed database.
Because the values are frozen at declaration, renaming the physical table or
column costs nothing: pin the old strings with :table and :column and
stored rows stay readable.
schema "payment_cards" do
field :pan, Payments.Encrypted.Binary, table: "cards", column: "pan"
endA pin written at the field wins over one written at the use, which wins
over the derivation. The field is where a pin usually belongs, because a
type module is shared by every field that names it while a pin is about one
column. Changing a declared value is the expensive direction - it
invalidates every row in the column and is a re-encryption migration. And
two fields must not share one declared pair, or each can decrypt the other's
bytes;
Encryptor.Ecto.Declarations.check_unique!/1 is the start-time check that
says so, and where a host calls it.
nil, and empty
nil dumps and loads as nil, with no encryption: a NULL column stays
NULL, because presence is already visible to anyone holding the database
and encrypting it would break every is_nil query a host has. An empty
binary is not nil and is encrypted - "" round-trips as ""
(decision 7).
Failures raise; they never return :error
cast/2 keeps the ordinary :error arm, because a non-binary handed to
this type genuinely is a validation failure, and cast/2 never encrypts.
dump/3 and load/3 have no :error arm at all (decision 6). An
Ecto.Type :error surfaces as a validation-shaped Ecto.ChangeError that
a changeset can catch and proceed past, writing the row without the value it
was supposed to protect - which is the wrong shape for an infrastructure
failure and a dangerous one for an integrity event.
| Condition | Exception |
|---|---|
| No tenant resolved | Encryptor.Ecto.MissingTenantError |
| The vault returned an encrypt error | Encryptor.Ecto.EncryptError |
| The vault returned a decrypt error, AAD mismatch included | Encryptor.Ecto.DecryptError |
| The vault reports missing required context keys | Encryptor.Ecto.MissingContextError |
A tenant: :none field names a :tenant-profile vault | Encryptor.Ecto.VaultProfileError |
No exception message, and no Inspect of one, carries plaintext, ciphertext
bytes or key material. That is structural rather than conventional: see
Encryptor.Ecto.Error.
A missing tenant is an error, deliberately
With tenant: :scope - the default - the tenant comes from
Encryptor.Ecto.Tenant, which the host sets at the edge of each unit of
work. A dump with nothing in scope raises Encryptor.Ecto.MissingTenantError
naming the table and the column (decision 5c). Not a default tenant, not a
nil tenant, not a global key, not a log line: the failure that choice
forbids is a row written under the wrong key because a background job
forgot, which is durable and unrecoverable in a way an exception on the
first test run is not.
Loads raise the same way (decision 5d). Loading with whatever context the
row implies and letting the AAD check fail would work, but a raise with a
legible message beats an authentication failure that reads like data
corruption. Where a tenant is in scope but is the wrong one, the AAD check
is the backstop: the read fails authentication and arrives as
Encryptor.Ecto.DecryptError, which is the anti-substitution property
working rather than a missing-scope error.
A global field: tenant: :none, and what it costs
tenant: :none declares a field global. It is written at the field, in the
schema, where a reviewer sees it next to the column it applies to, and it
asks no resolver anything: nothing is read from Encryptor.Ecto.Tenant, and
a dump with no tenant in scope is the ordinary case rather than an error.
A :none field's ciphertexts are not crypto-shreddable with a tenant
key. The tenant key is omitted from the vault call entirely, so those
bytes belong to the vault's single key and nothing else. Destroying one
tenant's key leaves every one of them readable, and removing that tenant's
data from a :none column is an ordinary delete rather than a
key-destruction. That is the trade the option is for - a lookup table, a
pricing tier, a feature flag payload that no tenant owns - and it is stated
here rather than left to the record, because the option is declared at the
field and its consequence is a compliance one.
defmodule Payments.Encrypted.Global do
use Encryptor.Ecto.Binary, vault: Payments.AppVault, tenant: :none
endA :none field must name a :single-profile vault
A :tenant-profile vault carries the tenant reference in its required
context set and refuses any operation without it, so "a tenant vault with
the pair omitted" is not a configuration that exists. A host with both kinds
of field runs two vaults: the per-tenant one its tenant-scoped fields point
at, and a second single-key one its global fields point at.
The rule is checked on the first dump/3 or load/3 of such a field, and
violating it raises Encryptor.Ecto.VaultProfileError naming the field, the
vault it named, and the profile that vault resolved to. It cannot be checked
while the declaration compiles: :context_profile is ordinary vault
configuration and arrives through layers - application environment,
start_link/1 options, init/1 - that do not exist at the compile time of
the vault module, let alone of a type module downstream of it. The
authoritative copy is the frozen Encryptor.Vault.Config the vault
publishes when it starts, and this type reads it from there rather than
keeping a second one.
Not queryable, and this package will not pretend otherwise
No equality lookup, no LIKE, no ordering, no unique index, no ON CONFLICT
target (decision 10). Ciphertext is non-deterministic, so equal?/3 compares
plaintext rather than stored bytes - otherwise every write would mark every
encrypted field changed (decision 9).
The migration window: :legacy
legacy: names the type module the host was reading this column with
before - a cloak_ecto type, a hand-rolled one - so that a column holding
both formats at once is readable for as long as the migration takes
(ADR-0001 acceptance amendment 4, ADR-0004 decision 4).
defmodule Payments.Encrypted.Binary do
use Encryptor.Ecto.Binary,
vault: Payments.Vault,
legacy: Payments.Cloak.Encrypted.Binary
endOrder, and what triggers the fallback
The primary load is attempted first, always. The legacy load is
attempted only when the vault refuses the stored bytes - the failure
that would otherwise raise Encryptor.Ecto.DecryptError. It is not
attempted for Encryptor.Ecto.MissingTenantError or
Encryptor.Ecto.MissingContextError: those are host misconfiguration, they
are loud on purpose, and a fallback that answered them with a successful
legacy read would convert a configuration bug into a silent year of
un-migrated rows (decision 4a).
The order is also what makes the window cheap. It costs one failed decrypt per legacy row and nothing at all per migrated row, and the population of legacy rows only shrinks.
Which error survives
If both loads fail, the exception raised is the primary
Encryptor.Ecto.DecryptError. The legacy attempt's reason travels in that
exception's non-contractual :engine field, redacted like everything else
in it, and must never be matched for control flow (decision 4b). Raising the
legacy error would report a self-expiring compatibility shim as the cause of
what is usually a genuine integrity event.
Nothing writes through legacy, ever
dump/3 has no legacy arm and never will (decision 4c). A value read
through the legacy path and written back is written in the new format, which
is what makes ordinary application traffic migrate rows on its own during
the window.
What the legacy module has to be
A module exporting load/1 and returning {:ok, value} | :error - the
Ecto.Type shape, which is what every cloak_ecto type and every
hand-rolled type already is. It is checked at the declaration rather than on
the first legacy row, the way Encryptor.Ecto.Map checks its :json.
A legacy scheme behind something else - an Ecto.ParameterizedType, a
sidecar service - is reachable by writing a one-function module around it,
and that is the supported route: this type constructs no params for a
foreign type and has no plan to construct them from. The migrator's from:
accepts more shapes than this option does, because ADR-0002 decision 3 has
it construct both sides' params itself and this type does not.
A zero-arity function returned by the legacy module is invoked once and its
result is the value, because deferred decryption is a real convention among
legacy types (cloak_ecto's closure: true) and a field that held the
function rather than the value would be a defect. This mirrors the rule
ADR-0004 decision 2 states for the migrator's Source contract.
Whatever the legacy module returns is the field's value as-is: this type
does not re-check it, and for Encryptor.Ecto.Map that means the legacy
module returns the map rather than bytes to deserialize.
The window is a security downgrade, and it is meant to end
While legacy: is set, a row that has not been rewritten yet is read under
the legacy scheme's rules: for a cloak_ecto host, with no encryption
context binding it to its row and no per-tenant key separation. No
migrated row is weakened; the guarantee is per-row until the rewrite
finishes (decision 5). Dropping legacy: is the last step of the runbook,
not a thing to remember.
Every load that falls through to the legacy path emits
:telemetry.execute([:encryptor_ecto, :legacy_load], %{count: 1},
%{table: "cards", column: "pan"})and the metadata set is closed at those two keys: no value, no bytes, no reason, no tenant. Widening it is a security review rather than a feature (decision 5). The pair is table and column precisely because the window is per-field: a host with twelve encrypted columns finishes eleven and still has one legacy reader open.
The counter is a convenience, not proof. ADR-0004's Q4 names the reason
and this documentation is the answer to it: a counter that has read zero for
a retention period is evidence about traffic, not about rows, so a table
with a cold partition nobody reads reports zero while still holding legacy
bytes. Encryptor.Ecto.Migrator.verify/2 over sample: :all is the primary
signal, and it is what a host drops legacy: on. Only the failed decrypt
that preceded a successful legacy read is counted - a load that failed
through both paths raises, which is louder than a counter and is not a
legacy row in the sense the window is about.
Summary
Functions
Defines an encrypted :binary type on the using module.
Casts a value on its way into a changeset. Never encrypts.
The encryption-context key names a declaration composes, sorted.
Encrypts a value on its way to the column.
Embeds as the cast value rather than the dumped one (decision 9).
Compares plaintext, never stored bytes.
Freezes a field's declared context, merging the declaration's options with
the :schema and :field Ecto supplies.
Decrypts the stored bytes on their way out of the column.
The same load, saying which of the two readers answered it.
The column type, which is :binary whatever the plaintext was.
Checks a declaration's option set while the declaring module compiles.
The same check, on behalf of a type built over this one.
Types
@type load_arm() :: :primary | :legacy
Which of the two readers answered a load.
:primary is the vault; :legacy is the module named by :legacy, and the
value it returned is that module's, unchecked by this one.
@type opts() :: [ vault: module(), tenant: :scope | :none | module(), context: %{optional(String.t()) => String.t()}, legacy: module(), table: String.t(), column: String.t() ]
The options use Encryptor.Ecto.Binary accepts.
@type params() :: %{ vault: module(), tenant: :scope | :none | module(), context: %{optional(String.t()) => String.t()}, table: String.t(), column: String.t(), legacy: module() | nil }
What init/2 freezes and every other callback receives.
A map rather than a keyword list, as Ecto.ParameterizedType suggests, so
the callbacks below pattern-match on it directly.
Functions
Defines an encrypted :binary type on the using module.
See the moduledoc for the option set; anything outside it raises here, while the host module is compiling.
Casts a value on its way into a changeset. Never encrypts.
Keeps the ordinary :error arm, because a non-binary handed to a binary
field really is a validation failure and belongs in a changeset's errors
(decision 6).
iex> Encryptor.Ecto.Binary.cast("4111111111111111", %{})
{:ok, "4111111111111111"}
iex> Encryptor.Ecto.Binary.cast(nil, %{})
{:ok, nil}
iex> Encryptor.Ecto.Binary.cast(:not_a_binary, %{})
:error
The encryption-context key names a declaration composes, sorted.
Public because a type built over this one can fail on the plaintext side of
the vault call - Encryptor.Ecto.Map's serializer does - and has to fill in
the same identifying fields on its own exception without recomposing the
context, which is the one thing that must not drift between the two.
iex> Encryptor.Ecto.Binary.context_keys(%{context: %{"purpose" => "pii"},
...> table: "cards", column: "pan"})
["column", "purpose", "table"]
Encrypts a value on its way to the column.
nil passes through unencrypted; everything else is handed to the vault
under the resolved tenant, with the declared table and column as encryption
context. There is no :error arm: the failure paths raise (decision 6).
Embeds as the cast value rather than the dumped one (decision 9).
iex> Encryptor.Ecto.Binary.embed_as(:json, %{})
:self
Compares plaintext, never stored bytes.
The same plaintext encrypts to different bytes every time, so a comparison over dumped values would mark every encrypted field changed on every write (decision 9).
iex> Encryptor.Ecto.Binary.equal?("4111111111111111", "4111111111111111", %{})
true
Freezes a field's declared context, merging the declaration's options with
the :schema and :field Ecto supplies.
The derivation runs once, here, and its results are explicit values in the
returned params rather than facts re-read from the live schema on every
operation (acceptance amendment 5). Renaming the physical table or field
while pinning :table or :column therefore leaves stored rows readable;
changing a declared value invalidates the column and is a data migration.
Raises when the table or the column cannot be resolved and was not supplied. A context-less encrypt is never performed (decision 4).
A :legacy module is resolved here too, and refused here if it cannot read
bytes: a declaration that names a legacy type which turns out not to export
load/1 is a migration window the host believes it has and does not, and
the first row is the wrong place to find that out.
Decrypts the stored bytes on their way out of the column.
nil passes through; everything else goes back to the vault unchanged, in
the same context the write composed. There is no :error arm here either:
a decrypt failure is an integrity event, not a validation error.
Where the declaration named :legacy, a vault refusal falls through to that
module rather than raising - see the moduledoc for the order, the trigger,
and which error survives when both fail.
The same load, saying which of the two readers answered it.
Public because a type built over this one can have work to do on the value
that only makes sense for one arm - Encryptor.Ecto.Map deserializes the
vault's plaintext and must not deserialize what a legacy module already
returned as a map. Every other caller wants load/3.
Raises exactly what load/3 raises, for exactly the same conditions.
@spec type(term()) :: :binary
The column type, which is :binary whatever the plaintext was.
Every one of these types stores the vault's bytes verbatim, so a migration has one column type to write and no length to guess (decisions 2 and 11).
iex> Encryptor.Ecto.Binary.type(%{})
:binary
Checks a declaration's option set while the declaring module compiles.
Returns the options unchanged when they are acceptable, so the macro above can assign the result straight into a module attribute.
iex> Encryptor.Ecto.Binary.validate_declaration!(Payments.Encrypted.Binary,
...> vault: Payments.Vault
...> )
[vault: Payments.Vault]
The same check, on behalf of a type built over this one.
Encryptor.Ecto.String and Encryptor.Ecto.Map share this option set rather
than re-implementing it, so a closed set stays closed in one place.
declared_by is the macro the host actually wrote, so the message names the
module a reader has to go and look up; extra_options is what that type adds
to the set (:json, for Map, and nothing for String).