StatifierPersistence.Ecto.Migrations.V07 (StatifierPersistence v0.22.0)

Copy Markdown View Source

V07 of the package DDL: what retention and retirement need in the schema (ADR-0012).

Five changes, one version:

  • a non-unique index on executions(content_hash), which the drained query groups under and which no earlier version provides - V01 indexes execution_id and V03 and V04 index metadata, so a count of the executions on one hash was a sequential scan of the host's whole executions table;
  • retired_at, a nullable utc_datetime_usec on charts;
  • retired_by, a nullable text column on charts;
  • identity_blob and chart_blob on charts made nullable.

The last two are what makes a retirement executable at all. V01 declares both blob columns null: false and no version between V01 and this one alters charts, so nulling a retired chart's bytes - which is what the removal is (ADR-0012 decision 6) - could not run against the schema as it stood.

Adapters other than Postgres

ALTER COLUMN is not a statement SQLite has, and ecto_sqlite3 raises ArgumentError from modify/3 rather than emitting one. Dropping a NOT NULL there means rebuilding the table and copying every row into the copy, which is not something this package's DDL does to a host's data - V06 records the same posture for the rename.

So the two modify changes are guarded, the way V03 guards its Postgres-only index, and the index and the two new columns are created on every backend. The consequence is worth stating plainly: on a backend that is not Postgres the blob columns keep null: false, so a retirement cannot null them and fails on the constraint. The chart doors, the drained query and the pin counting are unaffected; retiring a chart is a Postgres capability under this version, and a host on another backend that needs it alters the two columns in a migration of its own.

Rolling back over a retired chart is refused

down/1 reverses all five changes, and the reversal of the last two is the one that cannot always be done. A retired chart's row is still there - that is what a tombstone is - and its identity_blob and chart_blob are NULL, because the retirement removed the bytes. There are no bytes to put back, so restoring null: false over such a row is impossible, and the two answers that do not involve inventing data are to fail the rollback or to leave the constraint off and report nothing.

This version fails it, loudly, before it has changed anything: down/1 asks first whether any charts row carries a retired_at, and raises naming the table and the count when one does. A host that means to roll back past this version deletes the tombstoned rows itself first - the charts they stood for are gone either way, and StatifierPersistence.Storage.save_chart/3 refuses to revive one (ADR-0012 decision 6) - and runs the rollback again.

The check is the same on every backend. What it asks about is the tombstone, not the column definition, so the rule does not vary with what up/1 was able to do on that backend.

The probe runs late

The existence and tombstone queries go through Ecto.Migration.execute/1's function form rather than running in the body of down/1, for the reason V04 and V06 record: Ecto's migration runner queues a migration's commands and runs them at the end, so a query issued from the body reaches the database out of order with the DDL around it. A queued function runs in order - here, before the removals it guards.

Summary

Functions

Reverses all five changes, refusing first when any charts row is tombstoned - see the moduledoc.

Adds the executions(content_hash) index, the two tombstone columns on charts, and - on Postgres - the two nullable blob columns, per config.

Functions

down(config)

@spec down(StatifierPersistence.Ecto.Config.t()) :: :ok

Reverses all five changes, refusing first when any charts row is tombstoned - see the moduledoc.

up(config)

Adds the executions(content_hash) index, the two tombstone columns on charts, and - on Postgres - the two nullable blob columns, per config.