Migrations

Copy Markdown View Source

AshClickhouse generates ClickHouse DDL directly from your Ash resources. There is no separate migration DSL — the table schema is derived from the resource's attributes and clickhouse DSL options.

Tasks

AshClickhouse integrates with both the standard Ash tasks and its own named tasks. The Ash tasks (mix ash.codegen and mix ash.migrate) are discovered via AshClickhouse.DataLayer.Extension, which implements Spark.Dsl.Extension, so the Ash tasks and its dedicated tasks are:

mix ash.codegen             # print pending DDL (see below)
mix ash.migrate             # CREATE TABLE (+ ALTER) for each resource
mix ash_clickhouse.setup    # CREATE DATABASE IF NOT EXISTS for each repo
mix ash_clickhouse.migrate  # same as mix ash.migrate

mix ash.migrate is equivalent to mix ash_clickhouse.migrate, and mix ash.codegen prints the DDL that would be applied (without touching the database). The --check flag exits with status 1 when there are pending changes, which is useful for CI.

mix ash_clickhouse.setup iterates over all AshClickhouse.Repo modules and calls create_database/0. If a repo has no :database configured, it targets ClickHouse's default database rather than a database literally named nil.

mix ash_clickhouse.migrate iterates over all resources that export __ash_clickhouse__/1, filters by migrate?/1 and matching repo/1, then:

  1. Runs Migration.generate_resource_cql/1 (CREATE TABLE IF NOT EXISTS)
  2. Runs Migration.alter_table_cql/2 (schema evolution for existing tables)
  3. Runs Migration.alter_indexes_cql/2 (adds missing data-skipping indexes)

Resources with migrate false are skipped. A resource whose clickhouse block forgets repo is skipped with an error (rather than being migrated into every configured database), and a resource that raises during DDL generation (e.g. an invalid default) is reported and skipped so a single bad resource does not abort the whole run.

Generated CREATE TABLE

AshClickhouse.Migration.create_table_cql/1 produces a statement like:

CREATE TABLE IF NOT EXISTS "my_app_dev"."users" (
  "id" UUID,
  "name" String,
  "email" String,
  "age" Int64,
  "inserted_at" DateTime64(6)
)
ENGINE = MergeTree()
ORDER BY (id)

The generated clauses are:

  • ENGINE = <engine> (default MergeTree())
  • PARTITION BY <partition_by> (if set)
  • ORDER BY (<order_by>) (required for most engines)
  • PRIMARY KEY (<primary_key>) (if set)
  • SETTINGS <settings> (if set)
  • INDEX <name> (<expression>) TYPE <type> GRANULARITY <n> (one per index in the clickhouse block)

The database is qualified only when a database is configured on the resource or repo.

Schema evolution

Because ClickHouse does not support transactional migrations the way relational databases do, alter_table_cql/2 emits ALTER TABLE statements to add new columns when the resource definition changes. Run mix ash_clickhouse.migrate again after changing attributes to apply the diff.

Data-skipping indexes

ClickHouse has no B-tree indexes; it uses data-skipping indexes declared in the table DDL. Declare them in the clickhouse block (the index macro may be repeated, and granularity defaults to 1):

clickhouse do
  table "events"
  repo MyApp.Repo
  order_by "id"

  index name: :idx_user_id, expression: "user_id", type: "bloom_filter"
  index name: :idx_created_at, expression: "created_at", type: "minmax", granularity: 4
end

mix ash_clickhouse.migrate emits the indexes in the CREATE TABLE statement for new tables, and issues ALTER TABLE ... ADD INDEX IF NOT EXISTS for existing tables that lack them. Index type is validated against a whitelist (minmax, set, bloom_filter, ngrambf_v1, tokenbf_v1) at compile time, so a typo fails at compile rather than at migrate. Index names must be unique within a resource — declaring two indexes with the same name: raises a compile-time ArgumentError.

Indexes are additive only. Changing an existing index's type or expression is not auto-applied — ADD INDEX IF NOT EXISTS no-ops on a name collision. Instead, the migrate task prints a warning (with the exact DROP INDEX + ADD INDEX SQL to run) when an existing index's stored definition differs from the DSL. To change a definition, manually run:

ALTER TABLE my_db.events DROP INDEX `idx_user_id`;
ALTER TABLE my_db.events ADD INDEX `idx_user_id` (user_id) TYPE bloom_filter GRANULARITY 1;

The expression comparison is best-effort (ClickHouse normalizes stored expressions), so treat a type mismatch as authoritative and double-check expression mismatches by hand before acting.

Type mapping

Column types are derived from Ash attribute types. See Types for the complete table.

Tips

  • Always set order_by — most ClickHouse engines require it.
  • Use partition_by (e.g. toYYYYMM(inserted_at)) for large time-series tables to improve pruning.
  • Keep migrate false on resources you manage outside AshClickhouse.