ExArrow.FlightSQL.Client (ex_arrow v0.6.0)

View Source

Arrow Flight SQL client for executing SQL queries against a remote server.

Delegates to the configured implementation module (see :flight_sql_client_impl in application config). The default implementation is backed by NIFs using the arrow-flight + tonic Rust crate stack.

Quick start

{:ok, client} = ExArrow.FlightSQL.Client.connect("localhost:32010")
{:ok, result} = ExArrow.FlightSQL.Client.query(client, "SELECT id, name FROM users")
result.num_rows  #=> 42

Connection

connect/1 accepts a "host:port" URI string. connect/2 accepts the same string plus a keyword options list.

TLS

:tls valueBehaviour
not set, loopback hostplaintext (auto)
not set, remote hostTLS with native OS certificate store (auto, secure default)
falseplaintext regardless of host
trueTLS with native OS certificate store
[ca_cert_pem: pem]TLS with a custom PEM-encoded CA certificate

Authentication

Pass credentials as gRPC metadata via the :headers option:

{:ok, client} = ExArrow.FlightSQL.Client.connect("dremio.example.com:32010",
  tls: true,
  headers: [{"authorization", "Bearer my-pat-token"}]
)

Queries

query/2 collects all result batches and returns a materialized ExArrow.FlightSQL.Result. Use stream_query/2 for large result sets that must be consumed lazily.

execute/2 runs DML (INSERT / UPDATE / DELETE / DDL) and returns the affected row count or :unknown when the server does not report one.

Compatibility

Designed for DuckDB Flight SQL server (v0.10+), DataFusion, Dremio, and other servers that implement the Arrow Flight SQL specification. End-to-end validation requires a live server; see the flight_sql_integration test tag.

Multi-endpoint FlightInfo responses (distributed queries) are not supported in v0.5.0 — query/2 returns {:error, %Error{code: :multi_endpoint}} in that case.

Transaction operations (BEGIN, COMMIT, ROLLBACK) are deferred to v0.6.0.

Summary

Functions

Close the connection and release native resources.

Connect to a Flight SQL server at the given URI.

Execute a DML or DDL statement.

List database schemas visible to the connected user.

Retrieve server capability and SQL dialect information.

List tables visible to the connected user.

Prepare a SQL query on the server and return a reusable statement handle.

Execute a SQL query and return a materialized result.

Execute a SQL query and return a materialized result, raising on failure.

Execute a SQL query and return a lazy stream of record batches.

Types

t()

@opaque t()

Functions

close(client)

@spec close(t()) :: :ok

Close the connection and release native resources.

In v0.5.0 the underlying gRPC channel is released when the client handle is garbage-collected. Calling close/1 explicitly is safe and idempotent, but does not eagerly close the channel.

connect(uri, opts \\ [])

@spec connect(
  String.t(),
  keyword()
) :: {:ok, t()} | {:error, ExArrow.FlightSQL.Error.t()}

Connect to a Flight SQL server at the given URI.

uri must be a "host:port" string, e.g. "localhost:32010". An explicit port is strongly recommended; a bare "host" string is accepted as a convenience and defaults to port 32010 (the Arrow Flight SQL convention).

Examples

{:ok, client} = ExArrow.FlightSQL.Client.connect("localhost:32010")

{:ok, client} = ExArrow.FlightSQL.Client.connect("dremio.example.com:32010",
  tls: true,
  headers: [{"authorization", "Bearer token"}]
)

execute(client, sql)

@spec execute(t(), String.t()) ::
  {:ok, non_neg_integer() | :unknown} | {:error, ExArrow.FlightSQL.Error.t()}

Execute a DML or DDL statement.

Returns {:ok, n} where n is the number of affected rows (non-negative integer), or {:ok, :unknown} when the server does not report a row count.

Returns {:error, %ExArrow.FlightSQL.Error{}} on failure.

Concurrency

Concurrent calls on the same client handle are serialised. Create separate handles with connect/2 for parallel workloads.

Examples

{:ok, 3}        = ExArrow.FlightSQL.Client.execute(client, "DELETE FROM t WHERE id < 4")
{:ok, :unknown} = ExArrow.FlightSQL.Client.execute(client, "CREATE TABLE t (id INT)")

get_db_schemas(client, opts \\ [])

@spec get_db_schemas(
  t(),
  keyword()
) :: {:ok, ExArrow.Stream.t()} | {:error, ExArrow.FlightSQL.Error.t()}

List database schemas visible to the connected user.

Returns a lazy ExArrow.Stream of record batches. The result schema follows the Arrow Flight SQL specification:

ColumnTypeDescription
catalog_nameutf8Catalog name (nullable)
db_schema_nameutf8Schema name

Options

  • :catalog — filter by exact catalog name (default: no filter)
  • :db_schema_filter — SQL LIKE pattern for schema names (default: no filter)

Server compatibility

Server support for filter parameters is optional. A server that does not implement GetDbSchemas will return {:error, %Error{code: :unimplemented}}.

Examples

{:ok, stream} = ExArrow.FlightSQL.Client.get_db_schemas(client)
batches = Enum.to_list(stream)

{:ok, stream} = ExArrow.FlightSQL.Client.get_db_schemas(client, catalog: "main")

get_sql_info(client)

@spec get_sql_info(t()) ::
  {:ok, ExArrow.Stream.t()} | {:error, ExArrow.FlightSQL.Error.t()}

Retrieve server capability and SQL dialect information.

Returns a lazy ExArrow.Stream of record batches. Each row encodes a single SqlInfo entry as defined by the Arrow Flight SQL specification. The result schema has two columns:

ColumnTypeDescription
info_nameuint32Numeric SqlInfo code
valuedense_union(...)Value — type depends on the info code

All available SqlInfo entries are returned. The exact set depends on the server; not all servers expose all codes.

Server compatibility

A server that does not implement GetSqlInfo will return {:error, %Error{code: :unimplemented}}.

Examples

{:ok, stream} = ExArrow.FlightSQL.Client.get_sql_info(client)
batches = Enum.to_list(stream)

get_tables(client, opts \\ [])

@spec get_tables(
  t(),
  keyword()
) :: {:ok, ExArrow.Stream.t()} | {:error, ExArrow.FlightSQL.Error.t()}

List tables visible to the connected user.

Returns a lazy ExArrow.Stream of record batches. The result schema follows the Arrow Flight SQL specification:

ColumnTypeDescription
catalog_nameutf8Catalog name (nullable)
db_schema_nameutf8Schema name (nullable)
table_nameutf8Table name
table_typeutf8Table type, e.g. "TABLE", "VIEW"

When :include_schema is true, an additional table_schema column containing the IPC-encoded Arrow schema of each table is also included.

Options

  • :catalog — filter by exact catalog name (default: no filter)
  • :db_schema_filter — SQL LIKE pattern for schema names (default: no filter)
  • :table_name_filter — SQL LIKE pattern for table names (default: no filter)
  • :table_types — list of type strings to include, e.g. ["TABLE", "VIEW"] (default: all types)
  • :include_schematrue to include IPC-encoded column schema in results (default: false)

Server compatibility

Server support for filter parameters is optional. A server that does not implement a particular filter may ignore it and return unfiltered results or return {:error, %Error{code: :unimplemented}}.

Examples

{:ok, stream} = ExArrow.FlightSQL.Client.get_tables(client)
batches = Enum.to_list(stream)

{:ok, stream} = ExArrow.FlightSQL.Client.get_tables(client,
  db_schema_filter: "public",
  table_types: ["TABLE"]
)

prepare(client, sql)

@spec prepare(t(), String.t()) ::
  {:ok, ExArrow.FlightSQL.Statement.t()} | {:error, ExArrow.FlightSQL.Error.t()}

Prepare a SQL query on the server and return a reusable statement handle.

Sends CreatePreparedStatement to the server, which parses and plans the query and returns an opaque handle. The handle can be executed one or more times with ExArrow.FlightSQL.Statement.execute/1 (for SELECT-like queries) or ExArrow.FlightSQL.Statement.execute_update/1 (for DML/DDL).

Returns {:ok, %ExArrow.FlightSQL.Statement{}} or {:error, %ExArrow.FlightSQL.Error{}}.

Concurrency

Concurrent calls on the same client handle are serialised. Create separate handles with connect/2 for parallel workloads.

Compatibility

Prepared statement support is optional in the Flight SQL specification. Servers that do not implement CreatePreparedStatement return {:error, %Error{code: :unimplemented}}.

Parameter binding (passing ? placeholders with Arrow data) is not supported in v0.5.0.

Examples

{:ok, stmt} = ExArrow.FlightSQL.Client.prepare(client, "SELECT * FROM t")
{:ok, stream} = ExArrow.FlightSQL.Statement.execute(stmt)
batches = Enum.to_list(stream)

# Re-execute the same statement without re-preparing
{:ok, stream2} = ExArrow.FlightSQL.Statement.execute(stmt)

query(client, sql)

@spec query(t(), String.t()) ::
  {:ok, ExArrow.FlightSQL.Result.t()} | {:error, ExArrow.FlightSQL.Error.t()}

Execute a SQL query and return a materialized result.

Collects all record batches from the server before returning. For large result sets, prefer stream_query/2.

Returns {:ok, %ExArrow.FlightSQL.Result{}} or {:error, %ExArrow.FlightSQL.Error{}}.

Concurrency

Concurrent calls on the same client handle are serialised — the underlying gRPC client requires exclusive access per call. For parallel queries, create separate client handles with connect/2.

Examples

{:ok, result} = ExArrow.FlightSQL.Client.query(client, "SELECT * FROM t")
result.num_rows   #=> 100
result.schema     #=> %ExArrow.Schema{...}

query!(client, sql)

@spec query!(t(), String.t()) :: ExArrow.FlightSQL.Result.t()

Execute a SQL query and return a materialized result, raising on failure.

Raises ExArrow.FlightSQL.Error if the query fails.

Examples

result = ExArrow.FlightSQL.Client.query!(client, "SELECT * FROM t")

stream_query(client, sql)

@spec stream_query(t(), String.t()) ::
  {:ok, ExArrow.Stream.t()} | {:error, ExArrow.FlightSQL.Error.t()}

Execute a SQL query and return a lazy stream of record batches.

Returns {:ok, %ExArrow.Stream{}} where the stream is consumed one batch at a time via ExArrow.Stream.next/1. The gRPC connection remains open until the stream is exhausted or the stream resource is garbage-collected.

Prefer this over query/2 for large result sets.

Concurrency

Concurrent calls on the same client handle are serialised — the underlying gRPC client requires exclusive access per call. For parallel queries, create separate client handles with connect/2.

Examples

{:ok, stream} = ExArrow.FlightSQL.Client.stream_query(client, "SELECT * FROM large_table")
schema = ExArrow.Stream.schema(stream)
ExArrow.Stream.to_list(stream)  # collect all — or iterate lazily with next/1