ChDriver.
Copy Markdown
Decodes ClickHouse's "Native" block format — the columns and rows of a query result, once any compression envelope has been stripped away.
This is internal to the driver's decoding pipeline. A block carries its
column names, types, and row data; decode_block/1 turns that into
%{columns: [{name, type}], rows: [[term]]}.
Only a pragmatic subset of ClickHouse's type system is supported — see
ChDriver.Types.Registry for the scalar types and
ChDriver.Protocol.Block.Wrappers for compound types like Array,
Map, and Nullable. If you're adding support for a new ClickHouse
type, ARCHITECTURE.md has the map of which module owns what.
Serialization prefixes are hoisted to the front of the column
Some types carry a fixed-size serialization prefix ahead of their row
data: LowCardinality(T) an 8-byte dictionary key version, and
Variant(...) an 8-byte discriminator mode. The non-obvious part is
that ClickHouse writes every prefix in a column's type tree before
any of that column's data, not immediately before the sub-column the
prefix belongs to.
So Array(LowCardinality(String)) is [LC key version][array offsets][LC dictionary + indexes], not [array offsets][LC key version][LC dictionary + indexes]. Reading the prefix inline (where
the nesting would suggest) consumes the first 8 bytes of the offsets
instead, which silently mis-splits the rows rather than failing loudly.
That's why decoding is two-phase: decode_prefixes/2 walks the type
tree depth-first and consumes every prefix up front, then
decode_column_data/4 decodes the data with those prefix values passed
back in, popping them in the same depth-first order they were read.
A zero-row block carries no prefixes at all, so phase one is skipped
entirely in that case.
Summary
Functions
Decodes a Native block (everything after the external table name): BlockInfo, column count, row count, and each column's name/type/data.
Decodes num_rows rows of type from binary, consuming any hoisted
serialization prefixes from prefixes (in the depth-first order
decode_prefixes/2 produced them).
Decodes a Data or ProfileEvents packet body: the external table name, followed by a Native block.
Consumes the hoisted serialization prefixes for type from the front of
binary (see the moduledoc for why they're all up front rather than
inline), returning {:ok, prefixes, rest} where prefixes is the
depth-first-ordered list of prefix values.
Functions
Decodes a Native block (everything after the external table name): BlockInfo, column count, row count, and each column's name/type/data.
Returns {:ok, %{columns: [{name, type}], rows: [[term]]}, rest},
{:incomplete, binary}, or {:error, reason}.
Decodes num_rows rows of type from binary, consuming any hoisted
serialization prefixes from prefixes (in the depth-first order
decode_prefixes/2 produced them).
Returns {:ok, values, rest, remaining_prefixes}.
@spec decode_data_packet(binary(), ChDriver.Protocol.Block.Compressed.method()) :: {:ok, map(), binary()} | {:incomplete, binary()} | {:error, term()}
Decodes a Data or ProfileEvents packet body: the external table name, followed by a Native block.
compression (:none (default) or :lz4) must match whatever was
negotiated for this query — when :lz4, the block is decompressed
before decoding.
Returns {:ok, %{table_name:, columns:, rows:}, rest},
{:incomplete, binary}, or {:error, reason}.
Consumes the hoisted serialization prefixes for type from the front of
binary (see the moduledoc for why they're all up front rather than
inline), returning {:ok, prefixes, rest} where prefixes is the
depth-first-ordered list of prefix values.
Types with no prefix of their own still recurse into their inner types,
since a nested LowCardinality/Variant anywhere in the tree
contributes one.