ergon_migrate (ergon v0.5.0)
View SourceErgon's schema migrations, driven by migraterl.
The whole schema lives in ordinary .sql files under priv/migrations/, applied
in source order:
bootstrap/ once extensions, CREATE SCHEMA ergon
functions/ on_change routines the schema depends on (attached as triggers)
schema/ once the tables, indexes, constraints, RLS, property graph
routines/ on_change routines that depend on the schema
cron/ always the pg_cron notifier tick
teardown/ - NOT a source; run by teardown/1Order is necessary in both directions. functions/ must precede schema/
because the four triggers there reference ergon.temporal_versioning(),
ergon.enforce_job_transition(), ergon.block_child(), and
ergon.unblock_children(). Those bodies name ergon.jobs themselves, which is
fine only because PL/pgSQL defers name resolution to run time; a LANGUAGE sql
body would be rejected before the table existed. routines/ must follow
schema/ for exactly that reason: ergon.enqueue and ergon.jobs_asof* name
ergon.jobs as a return type, and ergon.notify_pending_jobs has a
LANGUAGE sql body that PostgreSQL validates at CREATE time.
migraterl scans each source directory non-recursively, sorts its .sql files
lexically, and journals each one under its basename. Names must therefore be
unique across every source in the namespace, which is what the single
000001..000015 sequence is for: it fixes the order within a directory and
keeps the journal keys distinct across directories.
Classes
once scripts are applied one time, keyed by name. on_change scripts are
re-applied whenever their content hash changes, which is why every routine is
CREATE OR REPLACE: editing a function body is an edit to its own file rather
than a new migration. always scripts run on every pass and are never journaled.
Dollar quoting
migraterl hands each script whole to epgsql:squery/2 (the simple query
protocol) and lets PostgreSQL's own parser split the statements, so $$-quoted
PL/pgSQL bodies are fine in a plain .sql file. A runner that split on ;
itself could not tolerate them.
One caveat: migraterl substitutes $key$ tokens from its
variables option before executing, so a variable named foo would rewrite the
dollar-quote tag $foo$. opts/1 passes no variables and every body in
priv/migrations uses a bare $$.
Summary
Functions
Open a boot-time epgsql connection from the PG* environment variables.
Apply every pending script, in every namespace.
The migraterl namespace Ergon journals its scripts under.
Every namespace this runner applies, Ergon's first.
The options map passed to migraterl:migrate/2 for Ergon's own namespace.
Like opts/0 with per-key overrides, e.g. opts(#{txn => single}).
Ordered {Class, Directory} sources, resolved against ergon's priv dir.
Like sources/0, but rooted at an explicit priv directory.
The currently-applied journal state, per namespace.
Drop everything Ergon's migration set created, and forget it was applied.
Types
-type summary() :: migraterl:summary().
Functions
-spec connect() -> {ok, epgsql:connection()} | {error, term()}.
Open a boot-time epgsql connection from the PG* environment variables.
migraterl speaks epgsql, not pgo, so this is the one place Ergon opens an
epgsql connection at all. It is short-lived: open it, migrate, disconnect/1.
Everything at runtime goes through ergon_repo on the pgo pool.
-spec disconnect(epgsql:connection()) -> ok.
Apply every pending script, in every namespace.
Ergon's namespace goes first and host namespaces follow in the order they were
registered. That ordering is not incidental: a host table that references
ergon.jobs, or attaches ergon.temporal_versioning() as a trigger, needs those
to exist already.
-spec migrate(epgsql:connection()) -> {ok, [{binary(), summary()}]} | {error, term()}.
-spec namespace() -> binary().
The migraterl namespace Ergon journals its scripts under.
Every namespace this runner applies, Ergon's first.
A host registers its own migration directories through application environment, each under its own migraterl namespace:
{ergon, [{ergon_migrate, [
{extra_sources, [
#{namespace => ~"my_app",
sources => [{once, {priv, my_app, "migrations"}}]}
]}
]}]}The separate namespace is what makes this safe rather than merely tidy.
migraterl journals every script under its basename and checks ordering
against that journal, so sharing Ergon's namespace would put host filenames in
competition with 000001..000015 for sort position, and a basename that happened
to collide would mark a host script as already applied without ever running it.
Namespaces are independent journals with their own advisory lock, so neither can
happen and a host is free to number its scripts however it likes.
Source directories accept the forms ergon_path:resolve_root/1 understands, the
same ones ergon_sql's extra_roots takes.
-spec opts() -> map().
The options map passed to migraterl:migrate/2 for Ergon's own namespace.
Like opts/0 with per-key overrides, e.g. opts(#{txn => single}).
Defaults to txn => per_script, so a failing script rolls back on its own and
the scripts before it stay applied. on_out_of_order => error rather than
migraterl's warn default: a not-yet-applied once script sorting before one
already applied means the numbering was reused, and that should stop the run
rather than log.
Dry run: report what migrate/1 would apply, without applying it.
Answers one summary per namespace, Ergon's first, in the order migrate/1 would
apply them.
-spec plan(epgsql:connection()) -> {ok, [{binary(), summary()}]} | {error, term()}.
-spec sources() -> [{once | on_change | always, file:filename_all()}].
Ordered {Class, Directory} sources, resolved against ergon's priv dir.
-spec sources(file:filename_all()) -> [{once | on_change | always, file:filename_all()}].
Like sources/0, but rooted at an explicit priv directory.
The currently-applied journal state, per namespace.
-spec status(epgsql:connection()) -> {ok, [{binary(), [map()]}]} | {error, term()}.
-spec teardown() -> ok | {error, term()}.
Drop everything Ergon's migration set created, and forget it was applied.
Runs priv/migrations/teardown/drop.sql, then deletes the ergon namespace's
rows from migraterl.schema_journal so a subsequent migrate/1 replays from
scratch. Without the journal wipe the once scripts would be skipped and the
schema would never come back.
Host namespaces are deliberately left alone, unlike migrate/1 which applies
them all. A host's schema is not Ergon's to drop, and guessing at how to unwind
it would be worse than not trying: there is no teardown script to run and no way
to know what depends on those tables. A host that wants a full reset tears its
own schema down first, then calls this.
Two ways a host table gets dropped anyway, both worth knowing because both look like this function misbehaving:
- It is in the
ergonschema without meaning to be. Ergon connects as a role namedergon, and the defaultsearch_pathof"$user", publicresolves$userto theergonschema. So an unqualifiedCREATE TABLEin a host migration lands inside Ergon's schema and goes with it. Qualify host DDL, or set asearch_pathon the migration. - It depends on an Ergon type. A column typed
ergon.job_state, or a foreign key into an Ergon table, makes the host table a dependent of the schema, andDROP SCHEMA ergon CASCADEtakes dependents with it. That is correct cascade behaviour rather than something this function chooses.
Destructive. The dev and test reset path, not something to run against a database holding jobs.
-spec teardown(epgsql:connection()) -> ok | {error, term()}.