TiDB.Ecto.Migrations (tidb v0.1.0)

Migration macros for managing TiDB TiFlash replicas, vector indexes, and full-text indexes.

TiDB leverages its columnar storage engine (TiFlash) to accelerate vector search (e.g., HNSW indexes) and full-text search. The macros in this module provide convenient wrappers for generating the necessary ALTER TABLE DDL statements within Ecto migrations.

Example Migration

defmodule MyApp.Repo.Migrations.CreateArticles do
  use Ecto.Migration
  import TiDB.Ecto.Migrations

  def up do
    create table(:articles) do
      add :title, :string
      add :body, :text
      add :embedding, :vector, size: 768

      timestamps()
    end

    # Enable TiFlash replica
    enable_tiflash("articles", replicas: 1)

    # Create HNSW vector index using cosine distance
    vector_index("articles", "embedding", distance: :cosine)

    # Create full-text index
    fulltext_index("articles", "body", parser: :standard)
  end

  def down do
    drop table(:articles)
  end
end

Summary

Functions

disable_tiflash(table)

Disables TiFlash columnar replication on the specified table by setting replica count to 0.

enable_tiflash(table, args \\ [])

Enables TiFlash columnar replication on the specified table.

fulltext_index(table, column, args \\ [])

Adds a full-text search index to a table column with on-demand columnar replica creation.

vector_index(table, column, args \\ [])

Adds a vector index to a table column with on-demand columnar replica creation.

Functions

disable_tiflash(table)

(macro)

Disables TiFlash columnar replication on the specified table by setting replica count to 0.

Arguments

table - Table name as a string or atom.

Executed SQL

ALTER TABLE <table> SET TIFLASH REPLICA 0;

Examples

disable_tiflash("documents")

enable_tiflash(table, args \\ [])

(macro)

Enables TiFlash columnar replication on the specified table.

Arguments

table - Table name as a string or atom.
args - Optional keyword list of options:
- :replicas - Number of TiFlash replicas to configure (default: 1).

Executed SQL

ALTER TABLE <table> SET TIFLASH REPLICA <replicas>;

Examples

enable_tiflash("documents")
enable_tiflash("documents", replicas: 2)

fulltext_index(table, column, args \\ [])

(macro)

Adds a full-text search index to a table column with on-demand columnar replica creation.

Arguments

table - Table name as a string or atom.
column - Column name as a string or atom.
args - Keyword list of options:
- :parser - Full-text parser to use (required). Supported atoms:
  - :standard - Uses STANDARD built-in parser.
  - :multilingual - Uses MULTILINGUAL parser for CJK/multilingual text.
- :name - Custom index name (default: "<table_name>_<column_name>_index").

Executed SQL

ALTER TABLE <table> ADD FULLTEXT INDEX <index_name> (<column>) WITH PARSER <parser> ADD_COLUMNAR_REPLICA_ON_DEMAND;

Examples

# Standard full-text index
fulltext_index("articles", "content", parser: :standard)

# Multilingual full-text index with custom name
fulltext_index("articles", "content", parser: :multilingual, name: "ft_articles_content")

Errors

Raises ArgumentError if :parser is not :standard or :multilingual.

vector_index(table, column, args \\ [])

(macro)

Adds a vector index to a table column with on-demand columnar replica creation.

Arguments

table - Table name as a string or atom.
column - Column name as a string or atom.
args - Optional keyword list of options:
- :distance - Distance metric to index. Supported atoms:
  - :cosine - Uses VEC_COSINE_DISTANCE (default).
  - :l2 - Uses VEC_L2_DISTANCE.
- :using - Vector index algorithm (default: "HNSW").
- :name - Custom index name (default: "<table_name>_<column_name>_index").

Executed SQL

ALTER TABLE <table> ADD VECTOR INDEX <index_name> ((<distance>(<column>))) USING <using> ADD_COLUMNAR_REPLICA_ON_DEMAND;

Examples

# Default HNSW cosine index
vector_index("items", "embedding")

# L2 (Euclidean) distance index with custom name
vector_index("items", "embedding", distance: :l2, name: "custom_vec_idx")

Errors

Raises ArgumentError if :distance is not :cosine or :l2.