ChDriver (ch_driver v0.1.0)

Copy Markdown

Public API for the ClickHouse native-protocol driver: a DBConnection pool wrapping ChDriver.Connection's TCP handshake and query encoding.

Usage

{:ok, pool} = ChDriver.start_link(hostname: "localhost", port: 9000)
{:ok, %ChDriver.Result{columns: columns, rows: rows}} =
  ChDriver.query(pool, "SELECT number FROM system.numbers LIMIT 5")

This is the intended integration point for higher layers (e.g. Ecto.Adapters.ClickHouse) -- start_link/1, query/2,3,4, query!/2,3,4, and stream/2,3,4 are the whole public surface; everything else (ChDriver.DBConnection, ChDriver.Connection, ChDriver.Protocol) is wiring underneath it.

All ChDriver.Connection.connect/1 options (:hostname, :port, :database, :username, :password, :connect_timeout, :recv_timeout, :compression) are accepted by start_link/1 and forwarded to each pooled connection; standard DBConnection.start_link/2 pool options (:pool_size, :name, etc.) are also accepted.

:compression (:none (default) or :lz4) is opt-in wire compression for Data blocks -- off by default, so existing callers see unchanged behavior. It can also be overridden per call via query/4's opts. See ChDriver.Connection.connect/1 and ChDriver.Protocol.Messages.encode_query/2 for how it's negotiated with the server.

Summary

Functions

query(conn, statement, params \\ [], opts \\ [])

Runs statement (a raw SQL string, optionally containing ClickHouse {name:Type} parameter placeholders) against conn (a pool started by start_link/1, or any DBConnection-compatible connection reference) and returns {:ok, %ChDriver.Result{}} or {:error, reason}.

query!(conn, statement, params \\ [], opts \\ [])

Same as query/4, but returns the %ChDriver.Result{} directly on success and raises the underlying error instead of returning {:error, reason}.

start_link(opts \\ [])

Starts a DBConnection pool of ClickHouse native-protocol connections.

stream(conn, statement, params \\ [], opts \\ [])

Builds a %ChDriver.Stream{} that lazily executes statement and yields its rows one wire-protocol Data block at a time (each element is a %ChDriver.Result{} for that block, honoring opts[:decode_mapper] exactly like query/2,3,4), instead of buffering the whole result in memory the way query/2,3,4 does.

Functions

query(conn, statement, params \\ [], opts \\ [])

@spec query(DBConnection.conn(), binary(), list(), keyword()) ::
  {:ok, ChDriver.Result.t()} | {:error, Exception.t()}

Runs statement (a raw SQL string, optionally containing ClickHouse {name:Type} parameter placeholders) against conn (a pool started by start_link/1, or any DBConnection-compatible connection reference) and returns {:ok, %ChDriver.Result{}} or {:error, reason}.

params is a list of {name, raw_text} or {name, raw_text, escape_rounds} tuples binding statement's placeholders -- see ChDriver.Params.text/1/ChDriver.Params.escape_rounds/1 and ChDriver.Query's moduledoc.

query!(conn, statement, params \\ [], opts \\ [])

@spec query!(DBConnection.conn(), binary(), list(), keyword()) :: ChDriver.Result.t()

Same as query/4, but returns the %ChDriver.Result{} directly on success and raises the underlying error instead of returning {:error, reason}.

start_link(opts \\ [])

@spec start_link(keyword()) :: {:ok, pid()} | {:error, term()}

Starts a DBConnection pool of ClickHouse native-protocol connections.

Returns {:ok, pid} (the pool/pid to pass as conn to query/2,3) or {:error, reason}.

stream(conn, statement, params \\ [], opts \\ [])

@spec stream(DBConnection.conn(), binary(), list(), keyword()) :: ChDriver.Stream.t()

Builds a %ChDriver.Stream{} that lazily executes statement and yields its rows one wire-protocol Data block at a time (each element is a %ChDriver.Result{} for that block, honoring opts[:decode_mapper] exactly like query/2,3,4), instead of buffering the whole result in memory the way query/2,3,4 does.

conn must already be a checked-out %DBConnection{} -- exactly what DBConnection.stream/4 itself requires (its resource/5 helper only matches an already-checked-out struct) -- so stream/2,3,4 has to be called from inside DBConnection.run/3 or DBConnection.transaction/3, not directly against a pool pid:

{:ok, result} =
  DBConnection.run(pool, fn conn ->
    conn
    |> ChDriver.stream("SELECT number FROM system.numbers LIMIT 200000")
    |> Enum.take(5)
  end)

See ChDriver.Stream's moduledoc and ChDriver.DBConnection's cursor section for how the underlying handle_declare/4/handle_fetch/4/ handle_deallocate/4 cycle keeps this genuinely incremental (each block is only read off the socket when the consumer asks for it) rather than pre-buffering everything and merely presenting it lazily.