ExArrow.Sink.Parquet (ex_arrow v0.7.0)

Write an Arrow stream or batch list to a Parquet file.

A thin sink that consumes an ExArrow.Stream.t() (or a {schema, [batches]} tuple) and writes it to a Parquet file via ExArrow.Parquet.Writer.to_file/3. Emits a [:ex_arrow, :parquet, :write] telemetry event.

Example

{:ok, stream} = ExArrow.Stream.from_flight_sql(client, "SELECT * FROM t")
ExArrow.Sink.Parquet.write(stream, "/data/out.parquet")

Summary

Functions

write(source, path)

Write source to a Parquet file at path.

Functions

write(source, path)

@spec write(
  ExArrow.Stream.t()
  | {ExArrow.Schema.t(), [ExArrow.RecordBatch.t()]}
  | [ExArrow.RecordBatch.t()],
  Path.t()
) :: :ok | {:error, String.t()}

Write source to a Parquet file at path.

source may be:

an ExArrow.Stream.t() — schema is read with ExArrow.Stream.schema/1 and batches with ExArrow.Stream.to_list/1.
a {schema, [batches]} tuple.
a list of ExArrow.RecordBatch.t() — schema is taken from the first batch.

Returns :ok or {:error, message}.