ChDriver.Protocol (ch_driver v0.1.0)

Copy Markdown

ClickHouse native-protocol packet types relevant to the initial connection handshake.

The Hello exchange (ClientHello sent, ServerHello received) happens in plaintext -- unlike Query/Data packets, it is never wrapped in a ChDriver.Protocol.Block.Compressed compressed envelope.

This module owns packet-type dispatch (decode_packet/1), the ClientHello/ServerHello struct definitions, and revision-gating predicates. Per-message byte layouts -- the actual encoding/decoding of each packet body -- live in ChDriver.Protocol.Messages.

Summary

Functions

Whether this driver's advertised revision requires sending the post-ServerHello Addendum (see ChDriver.Protocol.Messages.encode_addendum/0).

Builds a ClientHello struct advertising this driver's identity, for the given default_database, user, and password.

Decodes a single server response packet from the front of binary.

Decodes a ServerHello packet from the front of binary, including its leading packet-type varint (expected to be 0).

Encodes the Addendum sent immediately after receiving ServerHello, when addendum_required?/0 is true. See ChDriver.Protocol.Messages.encode_addendum/0 for the wire layout and why this step is required.

Encodes a Cancel packet (Client packet type 3). See ChDriver.Protocol.Messages.encode_cancel/0 for the wire layout and why the caller must keep draining the socket after sending this.

Encodes a ClientHello struct into the plaintext bytes sent to open a connection: packet-type varint (0), then client name/version/revision and the default_database/user/password length-prefixed strings.

Encodes an empty Data packet (Client packet type 2): an empty external table, sent right after a Query packet to signal "no external tables / no input data" (required even for plain SELECTs). compression must match whatever was passed as encode_query/2's :compression opt for this query. See ChDriver.Protocol.Messages.encode_empty_data_packet/1 for the wire layout.

Encodes a Ping packet (Client packet type 4). No body -- just the packet-type varint. The server replies with a bare Pong (Server packet type 4, see @server_pong / decode_packet/1) and nothing else.

Encodes a Query packet (Client packet type 1) for query_string.

Functions

addendum_required?()

@spec addendum_required?() :: boolean()

Whether this driver's advertised revision requires sending the post-ServerHello Addendum (see ChDriver.Protocol.Messages.encode_addendum/0).

client_hello(default_database \\ "default", user \\ "default", password \\ "")

@spec client_hello(binary(), binary(), binary()) :: ChDriver.Protocol.ClientHello.t()

Builds a ClientHello struct advertising this driver's identity, for the given default_database, user, and password.

decode_packet(binary, compression \\ :none)

@spec decode_packet(binary(), ChDriver.Protocol.Block.Compressed.method()) ::
  {:ok, term(), binary()} | {:incomplete, binary()} | {:error, term()}

Decodes a single server response packet from the front of binary.

Returns {:ok, packet, rest}, {:incomplete, binary} if more bytes are needed (retry with more buffered data), or {:error, reason}.

packet is one of:

  • {:data, %{table_name:, columns:, rows:}} -- a Data packet (Server packet type 1) or a wire-identical ProfileEvents packet (type 14, tagged :profile_events instead of :data).
  • {:profile_events, %{table_name:, columns:, rows:}}
  • {:progress, %{read_rows:, read_bytes:, total_rows_to_read:, total_bytes_to_read:, written_rows:, written_bytes:, elapsed_ns:}}
  • :pong
  • :end_of_stream
  • {:exception, %ChDriver.Error{}}

Packet type discriminants are documented in the module-level comments above; per-type field layouts live in ChDriver.Protocol.Messages.

compression (:none (default) or :lz4) must match whatever was negotiated for this query via encode_query/2's :compression opt -- it only affects how Data packets' block bodies are decoded (routed through ChDriver.Protocol.Block.Compressed.decode/1 first when :lz4).

ProfileEvents (type 14) is deliberately excluded from that: verified live against ClickHouse 24.8 with compression negotiated on, the server sends ProfileEvents blocks in plain, un-enveloped Native format regardless of the negotiated compression setting -- unlike Data, it's written via its own dedicated NativeWriter straight to the raw output stream rather than through the query's maybe_compressed_out. Forcing :lz4 decoding onto it (as this driver's first cut at compression did) desyncs the stream right after the first ProfileEvents packet, since ChDriver.Protocol.Block.Compressed.decode/1 then waits forever for a compression envelope header that was never written. So ProfileEvents is always decoded as :none, independent of compression; every other packet type (Progress, ProfileInfo, Exception, EndOfStream, Pong) was already plain regardless.

decode_server_hello(binary)

@spec decode_server_hello(binary()) ::
  {:ok, ChDriver.Protocol.ServerHello.t(), binary()}
  | {:incomplete, binary()}
  | {:error, term()}

Decodes a ServerHello packet from the front of binary, including its leading packet-type varint (expected to be 0).

Returns {:ok, %ServerHello{}, rest}, {:incomplete, binary} if more bytes are needed, or {:error, reason} if the packet type doesn't match Hello (e.g. the server sent an Exception instead, most likely due to bad credentials).

encode_addendum()

@spec encode_addendum() :: iodata()

Encodes the Addendum sent immediately after receiving ServerHello, when addendum_required?/0 is true. See ChDriver.Protocol.Messages.encode_addendum/0 for the wire layout and why this step is required.

encode_cancel()

@spec encode_cancel() :: iodata()

Encodes a Cancel packet (Client packet type 3). See ChDriver.Protocol.Messages.encode_cancel/0 for the wire layout and why the caller must keep draining the socket after sending this.

encode_client_hello(hello)

@spec encode_client_hello(ChDriver.Protocol.ClientHello.t()) :: iodata()

Encodes a ClientHello struct into the plaintext bytes sent to open a connection: packet-type varint (0), then client name/version/revision and the default_database/user/password length-prefixed strings.

encode_empty_data_packet(compression \\ :none)

@spec encode_empty_data_packet(ChDriver.Protocol.Block.Compressed.method()) ::
  iodata()

Encodes an empty Data packet (Client packet type 2): an empty external table, sent right after a Query packet to signal "no external tables / no input data" (required even for plain SELECTs). compression must match whatever was passed as encode_query/2's :compression opt for this query. See ChDriver.Protocol.Messages.encode_empty_data_packet/1 for the wire layout.

encode_ping()

@spec encode_ping() :: iodata()

Encodes a Ping packet (Client packet type 4). No body -- just the packet-type varint. The server replies with a bare Pong (Server packet type 4, see @server_pong / decode_packet/1) and nothing else.

encode_query(query_string, opts \\ [])

@spec encode_query(
  binary(),
  keyword()
) :: iodata()

Encodes a Query packet (Client packet type 1) for query_string.

opts accepts:

  • :query_id -- defaults to "".
  • :params -- a list of {name :: binary, raw_text :: binary} pairs binding query_string's {name:Type} placeholders (see ChDriver.Params.text/1 for turning an Elixir value into raw_text and ChDriver.Params.escape_rounds/1 for the matching escape depth -- a bare 2-tuple defaults to the scalar depth of 2). Defaults to [].

See ChDriver.Protocol.Messages.encode_query/2 for the full wire layout.