QuackDB.Source (quackdb v0.4.0)

Copy Markdown View Source

Safe SQL fragment builders for DuckDB table-producing data sources.

DuckDB can query files, object stores, and lakehouse tables directly through table functions such as read_parquet/2, read_csv/2, read_json/2, read_xlsx/2, delta_scan/2, and iceberg_scan/2. This module builds those fragments with QuackDB's SQL literal formatting so callers do not need to manually interpolate paths or options.

These helpers return SQL fragments, not executable queries. Use them in raw SQL that is sent to QuackDB.query/4 or Ecto.Adapters.SQL.query/4:

source = QuackDB.Source.parquet("s3://bucket/events/*.parquet", hive_partitioning: true)
QuackDB.query!(conn, ["SELECT count(*) FROM ", source])

Options are emitted as DuckDB named parameters:

QuackDB.Source.csv("events.csv", header: true, columns: %{id: "INTEGER", name: "VARCHAR"})

Plain Elixir maps are formatted as DuckDB struct literals, which is the shape used by options such as columns. Use {:map, map} when a DuckDB MAP {...} literal is required.

Summary

Types

option_value()
path_or_paths()

Functions

csv(path_or_paths, options \\ [])

Builds a read_csv(...) table function fragment.

delta(path_or_paths, options \\ [])

Builds a delta_scan(...) table function fragment.

histogram_values(source, column, options \\ [])

Builds a histogram_values(...) table function fragment.

iceberg(path_or_paths, options \\ [])

Builds an iceberg_scan(...) table function fragment.

json(path_or_paths, options \\ [])

Builds a read_json(...) table function fragment.

parquet(path_or_paths, options \\ [])

Builds a read_parquet(...) table function fragment.

sample(source, options)

Wraps a source in a DuckDB USING SAMPLE subquery.

source?(value)

Returns true when a value looks like a QuackDB source table-function fragment.

table_function(function_name, path_or_paths, options \\ [])

Builds a DuckDB table-function fragment for a validated function name.

xlsx(path, options \\ [])

Builds a read_xlsx(...) table function fragment.

Types

option_value()

@type option_value() ::
  QuackDB.SQL.parameter()
  | atom()
  | %{optional(atom() | String.t() | integer()) => option_value()}
  | {:struct, keyword(option_value()) | map()}
  | {:map, map()}

path_or_paths()

@type path_or_paths() :: String.t() | [String.t()]

Functions

csv(path_or_paths, options \\ [])

@spec csv(
  path_or_paths(),
  keyword(option_value())
) :: String.t()

Builds a read_csv(...) table function fragment.

delta(path_or_paths, options \\ [])

@spec delta(
  path_or_paths(),
  keyword(option_value())
) :: String.t()

Builds a delta_scan(...) table function fragment.

histogram_values(source, column, options \\ [])

@spec histogram_values(String.t(), String.t() | atom(), keyword(option_value())) ::
  String.t()

Builds a histogram_values(...) table function fragment.

iceberg(path_or_paths, options \\ [])

@spec iceberg(
  path_or_paths(),
  keyword(option_value())
) :: String.t()

Builds an iceberg_scan(...) table function fragment.

json(path_or_paths, options \\ [])

@spec json(
  path_or_paths(),
  keyword(option_value())
) :: String.t()

Builds a read_json(...) table function fragment.

parquet(path_or_paths, options \\ [])

@spec parquet(
  path_or_paths(),
  keyword(option_value())
) :: String.t()

Builds a read_parquet(...) table function fragment.

sample(source, options)

@spec sample(
  String.t(),
  keyword()
) :: String.t()

Wraps a source in a DuckDB USING SAMPLE subquery.

source?(value)

@spec source?(term()) :: boolean()

Returns true when a value looks like a QuackDB source table-function fragment.

table_function(function_name, path_or_paths, options \\ [])

@spec table_function(String.t(), path_or_paths(), keyword(option_value())) ::
  String.t()

Builds a DuckDB table-function fragment for a validated function name.

xlsx(path, options \\ [])

@spec xlsx(
  String.t(),
  keyword(option_value())
) :: String.t()

Builds a read_xlsx(...) table function fragment.