Versioned migrations for Arcana's core tables.

Host applications run one migration and never write Arcana's DDL by hand:

defmodule MyApp.Repo.Migrations.AddArcana do
  use Ecto.Migration

  def up, do: Arcana.Migration.up(dimensions: 384)
  def down, do: Arcana.Migration.down()
end

Later upgrades need no new migration content, only a new migration that calls up/1 again. Each release adds version steps; up/1 applies every step between the version recorded in your database and the target.

Options

• :version - target version (defaults to the latest)
• :dimensions - required. Embedding dimensions for arcana_chunks.embedding. Must match Arcana.Embedder.dimensions/1 for the embedder you run, or storing a chunk fails. There is no default: the column can't be resized without rebuilding every vector in it, and a wrong guess stays invisible on a database that already has the table. mix arcana.install detects it from your configured embedder and writes it into the migration it generates
• :prefix - Postgres schema to install into (defaults to the connection's current schema)
• :create_schema - whether to create :prefix when it is missing (defaults to true, ignored without a prefix)

Adoption

Installs created before this module existed have tables but no recorded version. Version 1 is written to converge them: tables are created only when absent, and columns added by later Arcana releases are added only when missing. Running it against an existing database changes nothing it already has. The one exception is a unique index whose shape is wrong: see "What converge verifies" below, which lists everything checked against the catalog rather than merely created when absent.

Where the version is recorded

The applied version is stored as the Postgres comment on arcana_documents, as exactly arcana:<n>:

SELECT obj_description('arcana_documents'::regclass);

Arcana owns that comment. Two consequences worth knowing before you point schema-documentation or data-catalog tooling at this table:

• recording a version replaces whatever comment is there, so a description you wrote on arcana_documents is lost on the next migration
• a comment Arcana doesn't recognise reads as version 0, the same as a fresh install, so up/1 re-runs the converge path (idempotent, and harmless) while down/1 declines to drop anything

Comment on any other table freely. Only arcana_documents is reserved.

What converge verifies

Adoption creates only what is absent, so most objects are checked by name alone. Three are verified against the catalog and corrected when they differ, because being merely present isn't enough:

• arcana_documents.collection_id's delete rule, since older templates emitted ON DELETE SET NULL
• the embedding column's dimension, which is compared against :dimensions rather than altered
• the unique index on arcana_collections(name) and on arcana_evaluation_test_case_chunks(test_case_id, chunk_id), since create_if_not_exists matches on the index name and an older template may have created one with a different shape

Everything else is create-if-absent. For the plain indexes that is deliberate: a differing one costs performance, not correctness.

Version history

• 1 - collections, documents, chunks and the evaluation tables