PhoenixKit.Migrations.ExpectedSchema.Object (phoenix_kit v2.0.0)

Copy Markdown View Source

The t/0 map shape emitted by PhoenixKit.Migrations.ExpectedSchema.objects/1 — one tracked schema object (a table, column, index, constraint, sequence, function, extension, or seed row) across the whole V01..@current_version chain.

This module is not the manifest itself. The manifest (PhoenixKit.Migrations.ExpectedSchema, tool-generated by dev_docs/squash/generate_baseline.exs, @moduledoc false) does not exist in this repository yet — it is produced from a real migrated scratch database (spec §5.1/§8.3) and lands with the squash PR (P3). Until then, go through PhoenixKit.Migrations.ExpectedSchema.Resolver to find a concrete implementation (the real manifest once generated, or a test fixture such as PhoenixKit.Test.FixtureExpectedSchema); this module documents the shape every such implementation returns and provides small pure helpers for consuming it correctly.

Field reference

  • :id — stable string identifier, "<class>:<key>". Format per class:

    classid formatexample
    :tabletable:<table>table:phoenix_kit_settings
    :columncolumn:<table>.<column>column:phoenix_kit_settings.module
    :indexindex:<name>index:phoenix_kit_email_logs_aws_message_id_uidx
    :constraintconstraint:<table>.<name>constraint:phoenix_kit_users.phoenix_kit_users_pkey
    :sequencesequence:<name>sequence:phoenix_kit_id_seq
    :functionfunction:<name>(<args>)function:uuid_generate_v7()
    :extensionextension:<name>extension:citext
    :seedseed:<table>:<business_key_value>seed:phoenix_kit_settings:time_zone

    Ids are stable across regenerations (deterministic emit order keys off {since, class, id}, never off list position) but are an internal convention, not a public stability guarantee beyond "unique within one manifest" — build lookups with Enum.find/2 / Map.new(objects, &{&1.id, &1}), never by parsing the string apart.

  • :class — see class/0. No :comment class and no :oban class (see "Deviations from spec §5.1" below).

  • :since — the version that introduced the object (first :added event in the generator's per-version diff pass). Always the same as the first tuple's version in :revisions.

  • :revisions — non-empty, ascending-by-version list of {as_of_version, shape} pairs, one entry per version the generator's diff pass observed the object's shape change (single-entry lists are the overwhelming majority; multi-entry objects are real and load-bearing — see "Multi-revision objects" below). shape is class-specific — see "Per-class shape keys" below.

  • :presence:required or :legacy_optional. See "Presence and legacy_optional" below.

  • :check{:catalog, spec} for every class except :seed (a structural catalog probe, spec §6.2 — never raw-text pg_dump diffing); :seed objects get a checked SQL string (a SELECT EXISTS (...) probe on the business key) because "does this row exist" has no catalog table to query. Never nil.

  • :createnil (never create — see below), a SQL string (fully additive: CREATE ... IF NOT EXISTS / ALTER TABLE ... ADD COLUMN IF NOT EXISTS / guarded ALTER TABLE ... ADD CONSTRAINT / INSERT ... ON CONFLICT DO NOTHING / INSERT ... WHERE NOT EXISTS), or {:helper, helper_call} for creation that has to run through Elixir (the uuid_generate_v7() function via PhoenixKit.Migrations.Postgres.Helpers.ensure_uuid_v7_function/1, or best-effort optional-module seeding — see "Helper creates" below).

  • :backfillnil or :default. Repair may only backfill a column it itself just added, from that column's own declared default (spec D7/§6.2) — this field flags exactly the columns where that applies (not_null: true and a non-nil default on the selected shape). It is computed independently of :presence; for a :legacy_optional column it is inert (:create is already nil, so the column is never added, so nothing is ever backfilled) but may still read :default.

:check/:create reflect the NEWEST revision — read this before writing repair code

:check and :create are pre-computed once, by the generator, from List.last(revisions) — the newest/final shape, unconditionally, for every class. They are not re-derived per target database. For a single-revision object that is also the only shape that will ever exist, so it is a no-op distinction. For a multi-revision object it is not: a database whose version comment sits between two revisions must be healed against the comment-era shape, not the newest one, or repair would "fix" an already-correct old-but-current shape into a change that only a later delta module is allowed to perform (spec §6.1's scope rule; healing V137's dedup or V141's normalize additively, outside the chain, is exactly the class of bug the scope rule exists to prevent).

Use shape_at/2 — never the object's own :check/:create fields — for anything version-scoped:

iex> revisions = [{53, %{type: "character varying(50)"}}, {142, %{type: "character varying(120)"}}]
iex> object = %{id: "x", class: :column, since: 53, revisions: revisions,
...>            presence: :required, check: "...", create: "...", backfill: nil}
iex> Object.shape_at(object, 100)
%{type: "character varying(50)"}
iex> Object.shape_at(object, 142)
%{type: "character varying(120)"}
iex> Object.shape_at(object, 10)
nil

A healed create/check statement for an intermediate comment version has to be rebuilt from the selected shape_at/2 result using the same idioms the generator itself uses (bare ADD COLUMN IF NOT EXISTS "<col>" <type>..., etc.) — this module intentionally does not do that rebuilding (it is SQL generation, squarely the repair engine's job, not the manifest contract's); it only exposes the revision-selection primitive every such rebuild starts from. newest_shape/1 is the complementary "what does this look like at the tip of the chain, independent of any DB" accessor (final-state reporting, --adopt's "everything at floor" case, docs).

Multi-revision objects

role_permissions.module_key is the running example throughout the spec: created VARCHAR(50) at V53, widened to VARCHAR(120) at V142. Its revisions list carries both entries. A database whose comment is anywhere in [53..141] verifies CLEAN against the V53 shape (spec §6.1, S17) — it is not drift for that DB to still be VARCHAR(50); only phoenix_kit.update running the real V142 delta module is allowed to widen it, because the delta's own migration may carry a data operation (a narrowing ALTER COLUMN TYPE for the reverse direction, in this case) that the manifest does not and must not replay.

Presence and legacy_optional

:required objects are the manifest's normal case: present in the incrementally-upgraded end-state, and repair creates them when missing (and the version's since-gate is satisfied). :legacy_optional objects mark the spec §3.7 bimodality: a handful of objects the OLD chain's fresh single-run end-state and its incrementally-upgraded end-state disagree about (an immediate-query guard racing a buffered execute, not a deliberate feature — users.preferred_locale is the canonical instance). Verify reports their presence either way at info level; repair never creates them (their :create is unconditionally nilvalid?/1 enforces this pairing). Finding one is not drift; synthesizing one that was never there is not a bug either.

Helper creates

{:helper, {mod, fun, args}} covers creation that cannot be expressed as a single SQL statement:

  • uuid_generate_v7(){Helpers, :ensure_uuid_v7_function, [:prefix]}. The literal atom :prefix inside args is a materialization placeholder, not a real argument — materialize/2 (called by every conformant objects/1, spec §5.1) substitutes it for the actual runtime prefix. Any other atom in args is passed through unchanged (so a literal atom argument stays representable, at the cost of :prefix itself being unusable as a literal argument value — the real generator makes the same trade-off).
  • Best-effort system email-template seeding ({Mix.Tasks.PhoenixKit.SeedTemplates, :run, [["--quiet"]]} and similar) — the v15/v31 lineage. These MFAs commonly name optional modules that legitimately do not exist in a given install (Mix tasks are stripped from production releases; the emails module may not be installed at all) — this is the expected common case, not a failure mode. A conformant executor invokes them via Code.ensure_loaded/1 + apply/3 + rescue, exactly like v15.ex/v31.ex do inline today, and treats "module not found" as a silent no-op. valid?/1 deliberately does not check whether the referenced module/function exists — doing so would make every fixture and every real install without the optional module "invalid".

Per-class shape keys

shape() is intentionally map(), not a per-class struct — the generator (PhoenixKit.Squash.Generate.Catalog.snapshot/2) produces plain maps whose key set is a property of class(), and this module mirrors that (a closed per-class struct union would be more machinery than the generator itself uses, for no behavioral gain). Keys actually emitted per class:

  • :table, :extension%{} (no shape data; existence is the whole story).
  • :column%{type: String.t(), not_null: boolean(), default: String.t() | nil, pos: pos_integer()}. type is format_type/2 output (e.g. "character varying(120)", "uuid"); default is pg_get_expr/2 output, nil when there is none.
  • :index%{table: String.t(), unique: boolean(), method: String.t(), definition: String.t(), predicate: String.t() | nil, keys: [String.t()], opclasses: [String.t()]}.
  • :constraint%{type: String.t(), definition: String.t(), columns: [String.t()] | nil, foreign_table: String.t() | nil, foreign_columns: [String.t()] | nil, on_delete: String.t() | nil, on_update: String.t() | nil}. type is the raw pg_constraint.contype char ("p"/"u"/"f"/"c"/"x"); on_delete/on_update are only ever non-nil for type == "f".
  • :sequence%{data_type: String.t(), start: integer(), increment: integer(), min: integer(), max: integer(), cache: integer(), cycle: boolean()}.
  • :function%{returns: String.t(), language: String.t(), body_md5: String.t(), definition: String.t()}.
  • :seed%{key_column: String.t(), key_value: String.t(), values: %{String.t() => term()}}. values is keyed by column name (string keys, matching a captured information_schema row) and includes key_column among its own keys.

Deviations from the spec §5.1 sketch (the generator is ground truth)

Per the P2 task brief: the generator (dev_docs/squash/generate_baseline.exs, frozen from P1) is ground truth; this module follows it, not the spec pseudocode, wherever the two disagree. Recorded here so the disagreement survives past this task:

  1. No :comment class. Spec §5.1's Object.t() comment lists class: :extension | ... | :seed | :comment. The generator's Differ module never produces a :comment-class object — the version-marker COMMENT ON TABLE is explicitly "not an object" (the generator's own render_manifest/2 doc comment) and spec §6.1's class-ordering line itself excludes comment from the generic sliced set ("comment handled ONLY per §6.4, not as a generic sliced object"). The §5.1 pseudocode comment and §6.1's prose disagree with each other; this module follows §6.1 + the generator.
  2. :create can be nil. Spec §5.1 types it as sql | {:helper, mfa} (no nil arm). The generator emits create: nil for every :legacy_optional object (Emitter.object_create/2's first clause) — required for "repair never creates them" to be representable at all.

  3. :backfill excludes {:manual, sql_text}. Spec §5.1 types it as nil | :default | {:manual, sql_text}. Emitter.object_backfill/1 only ever returns nil or :default — there is no code path that produces a :manual backfill today. backfill/0 below is typed nil | :default to match; widening it back to include :manual is a generator change, not a behaviour change, should a future revision need it.
  4. The real generated module does not declare @behaviour. The Emitter's render_manifest/2 template emits def objects/1, def data_invariants/1, def chain_hash/0 directly, with no @behaviour PhoenixKit.Migrations.ExpectedSchema.Behaviour line — so the compiler's own callback-completeness check never runs against it. PhoenixKit.Migrations.ExpectedSchema.Resolver therefore verifies conformance structurally, at runtime (__info__(:functions) membership), which works whether or not the target module opts in to @behaviour. Hand-written implementations (fixtures, and any future hand-written override) should still declare @behaviour for the free compile-time check — PhoenixKit.Test.FixtureExpectedSchema does.
  5. data_invariants/0 is data_invariants/1 with a defaulted arg. Spec §5.1 shows def data_invariants() :: [...] — no prefix parameter. The generator emits def data_invariants(prefix \\ "public"), because assert strings carry the same "__SCHEMA__" token as :check/:create (e.g. the V114 invariant queries __SCHEMA__.phoenix_kit_settings) and are useless for a prefixed install without substitution. Elixir's default-argument sugar compiles both data_invariants/0 and data_invariants/1 from that one def, so the spec's literal arity-0 call still works — but the behaviour callback in this contract is declared at arity 1, matching what is actually load-bearing.

Summary

Types

nil (nothing to backfill) or :default (repair may set the column to its own declared default — spec D7/§6.2; only ever set for class: :column objects). See moduledoc deviation 3 for why {:manual, sql_text} is not a valid value here despite appearing in the spec §5.1 sketch.

The catalog-probe kind for check :: {:catalog, catalog_spec}. The same atoms as class/0 minus :seed (seed existence is checked by SQL string, not a catalog lookup — see check/0).

Structural probe spec for check :: {:catalog, catalog_spec}. Required keys depend on :kind: :table/:extension/:sequence need only :name; :column needs :table + :column; :index/:constraint need :table + :name; :function needs :name + :args.

A structural catalog probe, or (for :seed objects) a checked SQL string.

The object's category. Matches PhoenixKit.Squash.Generate.Differ.@singular exactly. Deliberately excludes :comment (see moduledoc deviation 1) and :oban (Oban objects are delegated to Oban.Migration, never manifested — spec §6.1; a manifested Oban snapshot would flag every host with a different-version Oban dependency as drifted).

nil (legacy_optional; never create) | additive SQL | a helper invocation.

An apply/3-shaped invocation triple: {module, function_name, args}. Not the built-in mfa() type, which pairs a function with its arity rather than its argument list — this carries the literal args passed to apply/3. The atom :prefix inside args is a materialization placeholder (see moduledoc "Helper creates"), substituted by materialize/2.

:required (the normal case) or :legacy_optional (spec §3.7 bimodal drift).

One entry in :revisions — the shape as of (i.e. from, inclusive) as_of_version.

A class-specific catalog shape — see the moduledoc's "Per-class shape keys" table. Left as a plain map() because the generator itself never gives these a struct; shape_at/2/newest_shape/1 are the supported way to pull one out of an object without hand-rolling the revision-selection.

t()

One tracked schema object. See the moduledoc for the full field reference.

Functions

Substitutes schema_token/0 for prefix throughout one object's :create, :check, and every :revisions shape — the per-object half of what a conformant objects/1 does to its whole raw list. prefix must already be normalized (see normalize_prefix/1); this function does not validate it again.

The prefix-embedded-name marker tokens (see the @name_marker_exempt/ @name_marker_always module attribute comment above) — exposed the same way schema_token/0 is, so a hand-written implementation (fixtures included) builds prefix-embedded-name shapes against these rather than hardcoded literals.

The newest (final, current-HEAD) revision shape — what the object's own :check/:create were pre-computed against, independent of any specific database's comment. Useful for final-state reporting (e.g. --adopt's "does this DB match the floor-level target" gate, or docs generation) where there is no specific DB version to select against.

Normalizes a manifest-facing prefix argument the way every conformant objects/1/data_invariants/1 implementation must: nil becomes "public"; anything else is validated via PhoenixKit.Migrations.Postgres.Helpers.validate_prefix!/1 (raises ArgumentError for a prefix that cannot be safely interpolated into SQL). Call this once, at the top of objects/1, before materializing any object.

The "__SCHEMA__" placeholder token every conformant manifest embeds in raw SQL (:check/:create strings, shape() string values, and PhoenixKit.Migrations.ExpectedSchema.DataInvariant's :assert) in place of the validated runtime prefix. materialize/2 substitutes it; hand-written implementations (fixtures included) should build their raw data against this same token rather than a hardcoded literal, so a rename here cannot silently desync them.

The shape as of as_of_version — the newest revision whose own as_of_version is <= as_of_version, or nil if the object does not exist yet at that version (equivalently: object.since > as_of_version).

Structural runtime conformance check for one object — the closest thing to a t/0 typespec assertion ExUnit can run (Dialyzer's type union is a static-analysis-time check, not something a unit test can invoke). Verifies field presence, coarse value shapes, and the two cross-field invariants P2 code relies on

Types

backfill()

@type backfill() :: nil | :default

nil (nothing to backfill) or :default (repair may set the column to its own declared default — spec D7/§6.2; only ever set for class: :column objects). See moduledoc deviation 3 for why {:manual, sql_text} is not a valid value here despite appearing in the spec §5.1 sketch.

catalog_kind()

@type catalog_kind() ::
  :extension | :function | :sequence | :table | :column | :index | :constraint

The catalog-probe kind for check :: {:catalog, catalog_spec}. The same atoms as class/0 minus :seed (seed existence is checked by SQL string, not a catalog lookup — see check/0).

catalog_spec()

@type catalog_spec() :: %{
  :kind => catalog_kind(),
  optional(:name) => String.t(),
  optional(:table) => String.t(),
  optional(:column) => String.t(),
  optional(:args) => String.t()
}

Structural probe spec for check :: {:catalog, catalog_spec}. Required keys depend on :kind: :table/:extension/:sequence need only :name; :column needs :table + :column; :index/:constraint need :table + :name; :function needs :name + :args.

check()

@type check() :: {:catalog, catalog_spec()} | String.t()

A structural catalog probe, or (for :seed objects) a checked SQL string.

class()

@type class() ::
  :extension
  | :function
  | :sequence
  | :table
  | :column
  | :index
  | :constraint
  | :seed

The object's category. Matches PhoenixKit.Squash.Generate.Differ.@singular exactly. Deliberately excludes :comment (see moduledoc deviation 1) and :oban (Oban objects are delegated to Oban.Migration, never manifested — spec §6.1; a manifested Oban snapshot would flag every host with a different-version Oban dependency as drifted).

create()

@type create() :: nil | String.t() | {:helper, helper_call()}

nil (legacy_optional; never create) | additive SQL | a helper invocation.

helper_call()

@type helper_call() :: {module(), atom(), [term()]}

An apply/3-shaped invocation triple: {module, function_name, args}. Not the built-in mfa() type, which pairs a function with its arity rather than its argument list — this carries the literal args passed to apply/3. The atom :prefix inside args is a materialization placeholder (see moduledoc "Helper creates"), substituted by materialize/2.

presence()

@type presence() :: :required | :legacy_optional

:required (the normal case) or :legacy_optional (spec §3.7 bimodal drift).

revision()

@type revision() :: {as_of_version :: pos_integer(), shape()}

One entry in :revisions — the shape as of (i.e. from, inclusive) as_of_version.

shape()

@type shape() :: map()

A class-specific catalog shape — see the moduledoc's "Per-class shape keys" table. Left as a plain map() because the generator itself never gives these a struct; shape_at/2/newest_shape/1 are the supported way to pull one out of an object without hand-rolling the revision-selection.

t()

@type t() :: %{
  id: String.t(),
  class: class(),
  since: pos_integer(),
  revisions: [revision(), ...],
  presence: presence(),
  check: check(),
  create: create(),
  backfill: backfill()
}

One tracked schema object. See the moduledoc for the full field reference.

Functions

materialize(object, prefix)

@spec materialize(t(), String.t()) :: t()

Substitutes schema_token/0 for prefix throughout one object's :create, :check, and every :revisions shape — the per-object half of what a conformant objects/1 does to its whole raw list. prefix must already be normalized (see normalize_prefix/1); this function does not validate it again.

Mirrors PhoenixKit.Squash.Generate.Emitter's private materialize_object/2/materialize_create/2/materialize_check/2/ materialize_shape/2 exactly — the real generated module bakes an equivalent copy of this same substitution directly into its own emitted source (the generator's template has no dependency on this library module, so it cannot delegate here; this is the reusable version for any hand-written implementation, PhoenixKit.Test.FixtureExpectedSchema included).

name_marker(atom)

@spec name_marker(:exempt | :always) :: String.t()

The prefix-embedded-name marker tokens (see the @name_marker_exempt/ @name_marker_always module attribute comment above) — exposed the same way schema_token/0 is, so a hand-written implementation (fixtures included) builds prefix-embedded-name shapes against these rather than hardcoded literals.

newest_shape(map)

@spec newest_shape(t()) :: shape()

The newest (final, current-HEAD) revision shape — what the object's own :check/:create were pre-computed against, independent of any specific database's comment. Useful for final-state reporting (e.g. --adopt's "does this DB match the floor-level target" gate, or docs generation) where there is no specific DB version to select against.

normalize_prefix(prefix)

@spec normalize_prefix(String.t() | nil) :: String.t()

Normalizes a manifest-facing prefix argument the way every conformant objects/1/data_invariants/1 implementation must: nil becomes "public"; anything else is validated via PhoenixKit.Migrations.Postgres.Helpers.validate_prefix!/1 (raises ArgumentError for a prefix that cannot be safely interpolated into SQL). Call this once, at the top of objects/1, before materializing any object.

schema_token()

@spec schema_token() :: String.t()

The "__SCHEMA__" placeholder token every conformant manifest embeds in raw SQL (:check/:create strings, shape() string values, and PhoenixKit.Migrations.ExpectedSchema.DataInvariant's :assert) in place of the validated runtime prefix. materialize/2 substitutes it; hand-written implementations (fixtures included) should build their raw data against this same token rather than a hardcoded literal, so a rename here cannot silently desync them.

shape_at(map, as_of_version)

@spec shape_at(t(), pos_integer()) :: shape() | nil

The shape as of as_of_version — the newest revision whose own as_of_version is <= as_of_version, or nil if the object does not exist yet at that version (equivalently: object.since > as_of_version).

This is the revision-selection primitive from spec §6.1's scope rule ("each [object] at the newest revision with as_of_version <= comment") and from §5.1's baseline-slicing (Emitter.shape_at/2, generalized here under a version-neutral name since P2 callers select against a DB's live comment, not necessarily a squash floor). See the moduledoc's ":check/:create reflect the NEWEST revision" section for why callers must go through this instead of reading the object's own :check/:create whenever length(revisions) > 1.

valid?(object)

@spec valid?(term()) :: boolean()

Structural runtime conformance check for one object — the closest thing to a t/0 typespec assertion ExUnit can run (Dialyzer's type union is a static-analysis-time check, not something a unit test can invoke). Verifies field presence, coarse value shapes, and the two cross-field invariants P2 code relies on:

  • revisions is a non-empty, strictly-ascending-by-version list of {pos_integer, map} pairs, and its first version equals since;
  • presence == :legacy_optional implies create == nil (spec §6.1: repair never creates known-bimodal drift — see moduledoc "Presence and legacy_optional").

Never raises — safe as an Enum.all?/2/Enum.filter/2 predicate over a whole objects/1 list. Deliberately does not check whether a {:helper, {mod, fun, _}} create's mod/fun actually exist — referencing an optional, possibly-absent module is the expected case, not a defect (see moduledoc "Helper creates").