Ecto. Adapters. ClickHouse. Migration
(clickhouse_adapter_ecto v0.2.0)
Copy Markdown
Validated builders for ClickHouse-specific migration column types that
have no direct Ecto.Migration.add/3 equivalent.
Ecto.Migration.add/3 rejects any Ecto.Type/Ecto.ParameterizedType
module as a column type, so a type like
Ecto.Adapters.ClickHouse.Types.FixedString can never be given to it
directly -- only atoms, quoted atoms, composite tuples, and
references(...) are accepted. These builders produce the quoted-atom
form add/3 does accept, with the parameter validated up front instead
of only surfacing as a ClickHouse DDL error at migration time:
add(:code, Ecto.Adapters.ClickHouse.Migration.fixed_string(16))
add(:status, Ecto.Adapters.ClickHouse.Migration.low_cardinality(:string))FixedString(N) additionally has a full Ecto.ParameterizedType --
Ecto.Adapters.ClickHouse.Types.FixedString -- for the schema side,
where add/3's restriction doesn't apply:
field :code, Ecto.Adapters.ClickHouse.Types.FixedString, size: 16LowCardinality(T) doesn't get one: it's transparent to callers
(decoded to the same Elixir value T would decode to on its own), so a
plain field :status, :string already works. Map(K, V) gets neither
a builder nor a ParameterizedType -- use the quoted atom directly:
add(:m, :"Map(String, UInt32)")Building a table's options: string
table/2's options: takes a single raw string for everything after the
column list -- engine, ORDER BY, PARTITION BY, SETTINGS k = v, ....
table_options/1 builds that string from a keyword list instead of
requiring migration authors to hand-interpolate and quote it (SETTINGS
in particular gets tedious once it accumulates several key/value pairs):
create table(:events, primary_key: false,
options: Ecto.Adapters.ClickHouse.Migration.table_options(
engine: "MergeTree",
partition_by: "toYYYYMM(inserted_at)",
order_by: "id"
)
) do
add :id, :id, primary_key: true
add :inserted_at, :utc_datetime
endIt also resolves {:system, "ENV_VAR"} settings values from the
environment at the time the helper runs (migration-run time, i.e.
mix ecto.migrate, not compile time) -- handy for credentials that
shouldn't be committed as a literal string in a migration file, like the
Kafka-engine example in Ecto.Adapters.ClickHouse.DDL's moduledoc:
execute("""
CREATE TABLE events_queue (id UInt64, payload String)
#{Ecto.Adapters.ClickHouse.Migration.table_options(
engine: "Kafka",
settings: [
kafka_broker_list: {:system, "KAFKA_BROKER_LIST"},
kafka_topic_list: "events",
kafka_group_name: "events_consumer",
kafka_format: "JSONEachRow"
]
)}
""")which renders (given KAFKA_BROKER_LIST=kafka:9092 in the environment
the migration runs in) as:
ENGINE = Kafka SETTINGS kafka_broker_list = 'kafka:9092', kafka_topic_list = 'events', kafka_group_name = 'events_consumer', kafka_format = 'JSONEachRow'
Summary
Types
A single :settings value for table_options/1.
Functions
Builds the quoted-atom migration type for FixedString(size).
Builds the quoted-atom migration type for LowCardinality(inner_type),
where inner_type is any Ecto type
Ecto.Adapters.ClickHouse.DDL's column_type!/1 already knows how to
map to a ClickHouse column type (e.g. :string, :integer, :uuid,
{:array, :string}).
Builds the options string table/2's options: expects (everything
after the column list: ENGINE, PARTITION BY, ORDER BY,
SETTINGS), from a keyword list instead of a hand-quoted raw string.
Types
A single :settings value for table_options/1.
A plain string is rendered single-quoted (ClickHouse String-typed
settings, e.g. kafka_broker_list); a number or boolean is rendered
unquoted (ClickHouse numeric/boolean-ish settings, e.g.
kafka_num_consumers). {:system, "ENV_VAR"} is resolved from the
environment at call time (migration-run time) via System.fetch_env!/1
and then rendered single-quoted, same as a literal string.
Functions
@spec fixed_string(pos_integer()) :: atom()
Builds the quoted-atom migration type for FixedString(size).
add(:code, Ecto.Adapters.ClickHouse.Migration.fixed_string(16))Raises ArgumentError if size is not a positive integer.
Builds the quoted-atom migration type for LowCardinality(inner_type),
where inner_type is any Ecto type
Ecto.Adapters.ClickHouse.DDL's column_type!/1 already knows how to
map to a ClickHouse column type (e.g. :string, :integer, :uuid,
{:array, :string}).
add(:status, Ecto.Adapters.ClickHouse.Migration.low_cardinality(:string))Raises ArgumentError (via column_type!/1) if inner_type isn't a
type this adapter's migration DDL knows how to map.
Builds the options string table/2's options: expects (everything
after the column list: ENGINE, PARTITION BY, ORDER BY,
SETTINGS), from a keyword list instead of a hand-quoted raw string.
Options
:engine(required) -- the engine name/clause, e.g."MergeTree"or a full engine expression like"Kafka".:partition_by-- rendered asPARTITION BY <value>.:order_by-- rendered asORDER BY <value>.:settings-- a keyword list ofkey: valuepairs, rendered asSETTINGS key1 = 'value1', key2 = value2, .... String values are single-quoted; numbers and booleans are not. A value can also be{:system, "ENV_VAR"}to interpolate an environment variable resolved when the migration runs -- see the moduledoc's Kafka example.
Clause order in the rendered string (ENGINE · PARTITION BY ·
ORDER BY · SETTINGS) matches ClickHouse's own CREATE TABLE clause
order; only clauses that were given are included.
iex> Ecto.Adapters.ClickHouse.Migration.table_options(engine: "MergeTree", order_by: "id")
"ENGINE = MergeTree ORDER BY id"
iex> Ecto.Adapters.ClickHouse.Migration.table_options(
...> engine: "MergeTree",
...> partition_by: "toYYYYMM(inserted_at)",
...> order_by: "id",
...> settings: [index_granularity: 8192]
...> )
"ENGINE = MergeTree PARTITION BY toYYYYMM(inserted_at) ORDER BY id SETTINGS index_granularity = 8192"Raises ArgumentError if :engine is missing, if an unrecognized
top-level option key is given, if :settings isn't a keyword list, if
a setting's value isn't a string/number/boolean/{:system, _}, or if a
{:system, "ENV_VAR"} setting's environment variable isn't set.