LanceDB.Source (LanceDB v0.2.0)

Copy Markdown View Source

Explicit input for creation, writes, merges and column backfills.

rows/2 consumes an enumerable of maps in bounded chunks. batches/2 consumes native Arrow batches without converting them to rows. Supply schema: when creating a table, either on the Source or as the creation operation's option; existing-table writes use the stored schema and reject incompatible explicitly supplied source fields. Conflicting source and creation schemas raise before input is consumed. Sources are lazy and run in the calling process. Reuse requires a repeatable enumerable. Commit policy belongs to the write operation, independently of chunk size.

Summary

Functions

Describes an enumerable of Batch values. Accepts an optional schema.

Describes row input. Options: schema and batch_size (default 1024).

Types

t()

@opaque t()

Functions

batches(data, opts \\ [])

@spec batches(Enumerable.t(), keyword()) :: t()

Describes an enumerable of Batch values. Accepts an optional schema.

rows(data, opts \\ [])

@spec rows(Enumerable.t(), keyword()) :: t()

Describes row input. Options: schema and batch_size (default 1024).