Parquex (parquex v0.3.0)

View Source

Reads and writes Parquet files through reusable object stores.

Finite helpers accept row or column maps. Continuous workflows use an explicit Parquex.Schema, bounded Parquex.Batch values and a Parquex.Writer or Parquex.Stream.

Summary

Functions

native_status()

Verifies that the packaged native boundary can load.

open_writer(store, key, schema, options \\ [])

Opens a bounded Parquet writer for one store key.

read(store, key, options \\ [])

Materializes all selected rows from one finite Parquet file.

schema(store, key, options \\ [])

Inspects one Parquet file through bounded metadata reads.

stream(store, key, options \\ [])

Opens a pull-based stream of bounded batches from one Parquet file.

write(store, key, input)

Writes finite rows or columns to one Parquet file, inferring its schema.

write(store, key, schema, input)

Writes finite rows or columns with an explicit schema to one Parquet file.

write(store, key, schema, input, options)

Functions

native_status()

@spec native_status() ::
  {:ok, %{api_version: pos_integer()}} | {:error, Parquex.Error.t()}

Verifies that the packaged native boundary can load.

open_writer(store, key, schema, options \\ [])

@spec open_writer(Parquex.Store.t(), String.t(), Parquex.Schema.t(), keyword()) ::
  {:ok, Parquex.Writer.t()} | {:error, Parquex.Error.t()}

Opens a bounded Parquet writer for one store key.

read(store, key, options \\ [])

@spec read(Parquex.Store.t(), String.t(), keyword()) ::
  {:ok, [map()]} | {:error, Parquex.Error.t()}

Materializes all selected rows from one finite Parquet file.

schema(store, key, options \\ [])

@spec schema(Parquex.Store.t(), String.t(), keyword()) ::
  {:ok, Parquex.Schema.t()} | {:error, Parquex.Error.t()}

Inspects one Parquet file through bounded metadata reads.

stream(store, key, options \\ [])

@spec stream(Parquex.Store.t(), String.t(), keyword()) ::
  {:ok, Parquex.Stream.t()} | {:error, Parquex.Error.t()}

Opens a pull-based stream of bounded batches from one Parquet file.

write(store, key, input)

@spec write(Parquex.Store.t(), String.t(), term()) ::
  {:ok, Parquex.Store.Metadata.t()} | {:error, Parquex.Error.t()}

Writes finite rows or columns to one Parquet file, inferring its schema.

write(store, key, schema, input)

@spec write(Parquex.Store.t(), String.t(), Parquex.Schema.t(), term()) ::
  {:ok, Parquex.Store.Metadata.t()} | {:error, Parquex.Error.t()}
@spec write(Parquex.Store.t(), String.t(), term(), keyword()) ::
  {:ok, Parquex.Store.Metadata.t()} | {:error, Parquex.Error.t()}

Writes finite rows or columns with an explicit schema to one Parquet file.

write(store, key, schema, input, options)

@spec write(Parquex.Store.t(), String.t(), Parquex.Schema.t(), term(), keyword()) ::
  {:ok, Parquex.Store.Metadata.t()} | {:error, Parquex.Error.t()}