Encryptor.Ecto.Binary (Encryptor.Ecto v0.2.0)

Copy Markdown View Source

An encrypted :binary field, declared the way cloak_ecto declares one.

A host writes one module per encrypted type and names it from the schema:

defmodule Payments.Encrypted.Binary do
  use Encryptor.Ecto.Binary, vault: Payments.Vault
end

defmodule Payments.Cards.Card do
  use Ecto.Schema

  schema "cards" do
    field :merchant_id, :string
    field :pan, Payments.Encrypted.Binary
  end
end

The generated module implements Ecto.ParameterizedType, so changesets, queries and Repo calls keep their ordinary form and the encryption happens in the type rather than at the call site (ADR-0001 decision 1).

The option set is closed

OptionRequiredMeaning
:vaultyesThe Encryptor.Vault module this type encrypts through
:tenantno:scope (default), :none, or a module implementing Encryptor.Ecto.TenantContext (see "A global field" below)
:contextnoStatic extra context pairs merged into every operation
:legacynoThe migration window's legacy type: load through it when the primary load fails (see "The migration window" below)
:table, :columnnoOverrides for the frozen declared context values, writable at the use or at the field

An unknown option raises while the host module compiles, and there is no Application environment fallback for :vault: the vault is named at the declaration or the module does not compile (decision 3). Every option that could weaken the encryption context is one nobody gets, which is what makes the closed set worth its inconvenience.

What goes in the encryption context

"table" and "column", derived once at declaration time from the schema's source and the field's name, plus whatever :context adds (decision 4). The tenant is not a context pair: it passes to the vault as key:, and the vault derives and injects "tenant_ref" itself (acceptance amendment 1). A declared "table" or "column" wins over a :context pair of the same name - the derived values are the anti-substitution property and a static pair cannot be allowed to shadow them.

Schema prefixes are deliberately absent: a prefix is a deployment-time placement decision, and binding it would make a ciphertext un-restorable into a differently-prefixed database.

Because the values are frozen at declaration, renaming the physical table or column costs nothing: pin the old strings with :table and :column and stored rows stay readable.

schema "payment_cards" do
  field :pan, Payments.Encrypted.Binary, table: "cards", column: "pan"
end

A pin written at the field wins over one written at the use, which wins over the derivation. The field is where a pin usually belongs, because a type module is shared by every field that names it while a pin is about one column. Changing a declared value is the expensive direction - it invalidates every row in the column and is a re-encryption migration. And two fields must not share one declared pair, or each can decrypt the other's bytes; Encryptor.Ecto.Declarations.check_unique!/1 is the start-time check that says so, and where a host calls it.

nil, and empty

nil dumps and loads as nil, with no encryption: a NULL column stays NULL, because presence is already visible to anyone holding the database and encrypting it would break every is_nil query a host has. An empty binary is not nil and is encrypted - "" round-trips as "" (decision 7).

Failures raise; they never return :error

cast/2 keeps the ordinary :error arm, because a non-binary handed to this type genuinely is a validation failure, and cast/2 never encrypts. dump/3 and load/3 have no :error arm at all (decision 6). An Ecto.Type :error surfaces as a validation-shaped Ecto.ChangeError that a changeset can catch and proceed past, writing the row without the value it was supposed to protect - which is the wrong shape for an infrastructure failure and a dangerous one for an integrity event.

ConditionException
No tenant resolvedEncryptor.Ecto.MissingTenantError
The vault returned an encrypt errorEncryptor.Ecto.EncryptError
The vault returned a decrypt error, AAD mismatch includedEncryptor.Ecto.DecryptError
The vault reports missing required context keysEncryptor.Ecto.MissingContextError
A tenant: :none field names a :tenant-profile vaultEncryptor.Ecto.VaultProfileError

No exception message, and no Inspect of one, carries plaintext, ciphertext bytes or key material. That is structural rather than conventional: see Encryptor.Ecto.Error.

A missing tenant is an error, deliberately

With tenant: :scope - the default - the tenant comes from Encryptor.Ecto.Tenant, which the host sets at the edge of each unit of work. A dump with nothing in scope raises Encryptor.Ecto.MissingTenantError naming the table and the column (decision 5c). Not a default tenant, not a nil tenant, not a global key, not a log line: the failure that choice forbids is a row written under the wrong key because a background job forgot, which is durable and unrecoverable in a way an exception on the first test run is not.

Loads raise the same way (decision 5d). Loading with whatever context the row implies and letting the AAD check fail would work, but a raise with a legible message beats an authentication failure that reads like data corruption. Where a tenant is in scope but is the wrong one, the AAD check is the backstop: the read fails authentication and arrives as Encryptor.Ecto.DecryptError, which is the anti-substitution property working rather than a missing-scope error.

A global field: tenant: :none, and what it costs

tenant: :none declares a field global. It is written at the field, in the schema, where a reviewer sees it next to the column it applies to, and it asks no resolver anything: nothing is read from Encryptor.Ecto.Tenant, and a dump with no tenant in scope is the ordinary case rather than an error.

A :none field's ciphertexts are not crypto-shreddable with a tenant key. The tenant key is omitted from the vault call entirely, so those bytes belong to the vault's single key and nothing else. Destroying one tenant's key leaves every one of them readable, and removing that tenant's data from a :none column is an ordinary delete rather than a key-destruction. That is the trade the option is for - a lookup table, a pricing tier, a feature flag payload that no tenant owns - and it is stated here rather than left to the record, because the option is declared at the field and its consequence is a compliance one.

defmodule Payments.Encrypted.Global do
  use Encryptor.Ecto.Binary, vault: Payments.AppVault, tenant: :none
end

A :none field must name a :single-profile vault

A :tenant-profile vault carries the tenant reference in its required context set and refuses any operation without it, so "a tenant vault with the pair omitted" is not a configuration that exists. A host with both kinds of field runs two vaults: the per-tenant one its tenant-scoped fields point at, and a second single-key one its global fields point at.

The rule is checked on the first dump/3 or load/3 of such a field, and violating it raises Encryptor.Ecto.VaultProfileError naming the field, the vault it named, and the profile that vault resolved to. It cannot be checked while the declaration compiles: :context_profile is ordinary vault configuration and arrives through layers - application environment, start_link/1 options, init/1 - that do not exist at the compile time of the vault module, let alone of a type module downstream of it. The authoritative copy is the frozen Encryptor.Vault.Config the vault publishes when it starts, and this type reads it from there rather than keeping a second one.

Not queryable, and this package will not pretend otherwise

No equality lookup, no LIKE, no ordering, no unique index, no ON CONFLICT target (decision 10). Ciphertext is non-deterministic, so equal?/3 compares plaintext rather than stored bytes - otherwise every write would mark every encrypted field changed (decision 9).

The migration window: :legacy

legacy: names the type module the host was reading this column with before - a cloak_ecto type, a hand-rolled one - so that a column holding both formats at once is readable for as long as the migration takes (ADR-0001 acceptance amendment 4, ADR-0004 decision 4).

defmodule Payments.Encrypted.Binary do
  use Encryptor.Ecto.Binary,
    vault: Payments.Vault,
    legacy: Payments.Cloak.Encrypted.Binary
end

Order, and what triggers the fallback

The primary load is attempted first, always. The legacy load is attempted only when the vault refuses the stored bytes - the failure that would otherwise raise Encryptor.Ecto.DecryptError. It is not attempted for Encryptor.Ecto.MissingTenantError or Encryptor.Ecto.MissingContextError: those are host misconfiguration, they are loud on purpose, and a fallback that answered them with a successful legacy read would convert a configuration bug into a silent year of un-migrated rows (decision 4a).

The order is also what makes the window cheap. It costs one failed decrypt per legacy row and nothing at all per migrated row, and the population of legacy rows only shrinks.

Which error survives

If both loads fail, the exception raised is the primary Encryptor.Ecto.DecryptError. The legacy attempt's reason travels in that exception's non-contractual :engine field, redacted like everything else in it, and must never be matched for control flow (decision 4b). Raising the legacy error would report a self-expiring compatibility shim as the cause of what is usually a genuine integrity event.

Nothing writes through legacy, ever

dump/3 has no legacy arm and never will (decision 4c). A value read through the legacy path and written back is written in the new format, which is what makes ordinary application traffic migrate rows on its own during the window.

What the legacy module has to be

A module exporting load/1 and returning {:ok, value} | :error - the Ecto.Type shape, which is what every cloak_ecto type and every hand-rolled type already is. It is checked at the declaration rather than on the first legacy row, the way Encryptor.Ecto.Map checks its :json.

A legacy scheme behind something else - an Ecto.ParameterizedType, a sidecar service - is reachable by writing a one-function module around it, and that is the supported route: this type constructs no params for a foreign type and has no plan to construct them from. The migrator's from: accepts more shapes than this option does, because ADR-0002 decision 3 has it construct both sides' params itself and this type does not.

A zero-arity function returned by the legacy module is invoked once and its result is the value, because deferred decryption is a real convention among legacy types (cloak_ecto's closure: true) and a field that held the function rather than the value would be a defect. This mirrors the rule ADR-0004 decision 2 states for the migrator's Source contract.

Whatever the legacy module returns is the field's value as-is: this type does not re-check it, and for Encryptor.Ecto.Map that means the legacy module returns the map rather than bytes to deserialize.

The window is a security downgrade, and it is meant to end

While legacy: is set, a row that has not been rewritten yet is read under the legacy scheme's rules: for a cloak_ecto host, with no encryption context binding it to its row and no per-tenant key separation. No migrated row is weakened; the guarantee is per-row until the rewrite finishes (decision 5). Dropping legacy: is the last step of the runbook, not a thing to remember.

Every load that falls through to the legacy path emits

:telemetry.execute([:encryptor_ecto, :legacy_load], %{count: 1},
  %{table: "cards", column: "pan"})

and the metadata set is closed at those two keys: no value, no bytes, no reason, no tenant. Widening it is a security review rather than a feature (decision 5). The pair is table and column precisely because the window is per-field: a host with twelve encrypted columns finishes eleven and still has one legacy reader open.

The counter is a convenience, not proof. ADR-0004's Q4 names the reason and this documentation is the answer to it: a counter that has read zero for a retention period is evidence about traffic, not about rows, so a table with a cold partition nobody reads reports zero while still holding legacy bytes. Encryptor.Ecto.Migrator.verify/2 over sample: :all is the primary signal, and it is what a host drops legacy: on. Only the failed decrypt that preceded a successful legacy read is counted - a load that failed through both paths raises, which is louder than a counter and is not a legacy row in the sense the window is about.

Summary

Types

Which of the two readers answered a load.

The options use Encryptor.Ecto.Binary accepts.

What init/2 freezes and every other callback receives.

Functions

Defines an encrypted :binary type on the using module.

Casts a value on its way into a changeset. Never encrypts.

The encryption-context key names a declaration composes, sorted.

Encrypts a value on its way to the column.

Embeds as the cast value rather than the dumped one (decision 9).

Compares plaintext, never stored bytes.

Freezes a field's declared context, merging the declaration's options with the :schema and :field Ecto supplies.

Decrypts the stored bytes on their way out of the column.

The same load, saying which of the two readers answered it.

The column type, which is :binary whatever the plaintext was.

Checks a declaration's option set while the declaring module compiles.

The same check, on behalf of a type built over this one.

Types

load_arm()

@type load_arm() :: :primary | :legacy

Which of the two readers answered a load.

:primary is the vault; :legacy is the module named by :legacy, and the value it returned is that module's, unchecked by this one.

opts()

@type opts() :: [
  vault: module(),
  tenant: :scope | :none | module(),
  context: %{optional(String.t()) => String.t()},
  legacy: module(),
  table: String.t(),
  column: String.t()
]

The options use Encryptor.Ecto.Binary accepts.

params()

@type params() :: %{
  vault: module(),
  tenant: :scope | :none | module(),
  context: %{optional(String.t()) => String.t()},
  table: String.t(),
  column: String.t(),
  legacy: module() | nil
}

What init/2 freezes and every other callback receives.

A map rather than a keyword list, as Ecto.ParameterizedType suggests, so the callbacks below pattern-match on it directly.

Functions

__using__(opts)

(macro)
@spec __using__(opts()) :: Macro.t()

Defines an encrypted :binary type on the using module.

See the moduledoc for the option set; anything outside it raises here, while the host module is compiling.

cast(value, params)

@spec cast(term(), term()) :: {:ok, binary() | nil} | :error

Casts a value on its way into a changeset. Never encrypts.

Keeps the ordinary :error arm, because a non-binary handed to a binary field really is a validation failure and belongs in a changeset's errors (decision 6).

iex> Encryptor.Ecto.Binary.cast("4111111111111111", %{})
{:ok, "4111111111111111"}

iex> Encryptor.Ecto.Binary.cast(nil, %{})
{:ok, nil}

iex> Encryptor.Ecto.Binary.cast(:not_a_binary, %{})
:error

context_keys(params)

@spec context_keys(params()) :: [String.t()]

The encryption-context key names a declaration composes, sorted.

Public because a type built over this one can fail on the plaintext side of the vault call - Encryptor.Ecto.Map's serializer does - and has to fill in the same identifying fields on its own exception without recomposing the context, which is the one thing that must not drift between the two.

iex> Encryptor.Ecto.Binary.context_keys(%{context: %{"purpose" => "pii"},
...>   table: "cards", column: "pan"})
["column", "purpose", "table"]

dump(value, dumper, params)

@spec dump(term(), function(), params()) :: {:ok, binary() | nil}

Encrypts a value on its way to the column.

nil passes through unencrypted; everything else is handed to the vault under the resolved tenant, with the declared table and column as encryption context. There is no :error arm: the failure paths raise (decision 6).

embed_as(format, params)

@spec embed_as(atom(), term()) :: :self

Embeds as the cast value rather than the dumped one (decision 9).

iex> Encryptor.Ecto.Binary.embed_as(:json, %{})
:self

equal?(left, right, params)

@spec equal?(term(), term(), term()) :: boolean()

Compares plaintext, never stored bytes.

The same plaintext encrypts to different bytes every time, so a comparison over dumped values would mark every encrypted field changed on every write (decision 9).

iex> Encryptor.Ecto.Binary.equal?("4111111111111111", "4111111111111111", %{})
true

init(declared, field_opts)

@spec init(keyword(), keyword()) :: params()

Freezes a field's declared context, merging the declaration's options with the :schema and :field Ecto supplies.

The derivation runs once, here, and its results are explicit values in the returned params rather than facts re-read from the live schema on every operation (acceptance amendment 5). Renaming the physical table or field while pinning :table or :column therefore leaves stored rows readable; changing a declared value invalidates the column and is a data migration.

Raises when the table or the column cannot be resolved and was not supplied. A context-less encrypt is never performed (decision 4).

A :legacy module is resolved here too, and refused here if it cannot read bytes: a declaration that names a legacy type which turns out not to export load/1 is a migration window the host believes it has and does not, and the first row is the wrong place to find that out.

load(value, loader, params)

@spec load(term(), function(), params()) :: {:ok, term()}

Decrypts the stored bytes on their way out of the column.

nil passes through; everything else goes back to the vault unchanged, in the same context the write composed. There is no :error arm here either: a decrypt failure is an integrity event, not a validation error.

Where the declaration named :legacy, a vault refusal falls through to that module rather than raising - see the moduledoc for the order, the trigger, and which error survives when both fail.

load_arm(value, params)

@spec load_arm(binary(), params()) :: {load_arm(), term()}

The same load, saying which of the two readers answered it.

Public because a type built over this one can have work to do on the value that only makes sense for one arm - Encryptor.Ecto.Map deserializes the vault's plaintext and must not deserialize what a legacy module already returned as a map. Every other caller wants load/3.

Raises exactly what load/3 raises, for exactly the same conditions.

type(params)

@spec type(term()) :: :binary

The column type, which is :binary whatever the plaintext was.

Every one of these types stores the vault's bytes verbatim, so a migration has one column type to write and no length to guess (decisions 2 and 11).

iex> Encryptor.Ecto.Binary.type(%{})
:binary

validate_declaration!(module, opts)

@spec validate_declaration!(
  module(),
  keyword()
) :: keyword()

Checks a declaration's option set while the declaring module compiles.

Returns the options unchanged when they are acceptable, so the macro above can assign the result straight into a module attribute.

iex> Encryptor.Ecto.Binary.validate_declaration!(Payments.Encrypted.Binary,
...>   vault: Payments.Vault
...> )
[vault: Payments.Vault]

validate_declaration!(module, opts, declared_by, extra_options)

@spec validate_declaration!(module(), keyword(), module(), [atom()]) :: keyword()

The same check, on behalf of a type built over this one.

Encryptor.Ecto.String and Encryptor.Ecto.Map share this option set rather than re-implementing it, so a closed set stays closed in one place. declared_by is the macro the host actually wrote, so the message names the module a reader has to go and look up; extra_options is what that type adds to the set (:json, for Map, and nothing for String).