V07 of the package DDL: what retention and retirement need in the schema (ADR-0012).
Five changes, one version:
- a non-unique index on
executions(content_hash), which the drained query groups under and which no earlier version provides - V01 indexesexecution_idand V03 and V04 indexmetadata, so a count of the executions on one hash was a sequential scan of the host's whole executions table; retired_at, a nullableutc_datetime_useconcharts;retired_by, a nullabletextcolumn oncharts;identity_blobandchart_blobonchartsmade nullable.
The last two are what makes a retirement executable at all. V01
declares both blob columns null: false and no version between V01
and this one alters charts, so nulling a retired chart's bytes -
which is what the removal is (ADR-0012 decision 6) - could not run
against the schema as it stood.
Adapters other than Postgres
ALTER COLUMN is not a statement SQLite has, and ecto_sqlite3
raises ArgumentError from modify/3 rather than emitting one.
Dropping a NOT NULL there means rebuilding the table and copying
every row into the copy, which is not something this package's DDL
does to a host's data - V06 records the same posture for the rename.
So the two modify changes are guarded, the way V03 guards its
Postgres-only index, and the index and the two new columns are
created on every backend. The consequence is worth stating plainly:
on a backend that is not Postgres the blob columns keep
null: false, so a retirement cannot null them and fails on the
constraint. The chart doors, the drained query and the pin counting
are unaffected; retiring a chart is a Postgres capability under this
version, and a host on another backend that needs it alters the two
columns in a migration of its own.
Rolling back over a retired chart is refused
down/1 reverses all five changes, and the reversal of the last two
is the one that cannot always be done. A retired chart's row is still
there - that is what a tombstone is - and its identity_blob and
chart_blob are NULL, because the retirement removed the bytes.
There are no bytes to put back, so restoring null: false over such
a row is impossible, and the two answers that do not involve
inventing data are to fail the rollback or to leave the constraint
off and report nothing.
This version fails it, loudly, before it has changed anything: down/1
asks first whether any charts row carries a retired_at, and
raises naming the table and the count when one does. A host that
means to roll back past this version deletes the tombstoned rows
itself first - the charts they stood for are gone either way, and
StatifierPersistence.Storage.save_chart/3 refuses to revive one
(ADR-0012 decision 6) - and runs the rollback again.
The check is the same on every backend. What it asks about is the
tombstone, not the column definition, so the rule does not vary with
what up/1 was able to do on that backend.
The probe runs late
The existence and tombstone queries go through
Ecto.Migration.execute/1's function form rather than running in the
body of down/1, for the reason V04 and V06 record: Ecto's migration
runner queues a migration's commands and runs them at the end, so a
query issued from the body reaches the database out of order with the
DDL around it. A queued function runs in order - here, before the
removals it guards.
Summary
Functions
Reverses all five changes, refusing first when any charts row is
tombstoned - see the moduledoc.
Adds the executions(content_hash) index, the two tombstone columns
on charts, and - on Postgres - the two nullable blob columns, per
config.
Functions
@spec down(StatifierPersistence.Ecto.Config.t()) :: :ok
Reverses all five changes, refusing first when any charts row is
tombstoned - see the moduledoc.
@spec up(StatifierPersistence.Ecto.Config.t()) :: :ok
Adds the executions(content_hash) index, the two tombstone columns
on charts, and - on Postgres - the two nullable blob columns, per
config.