LanceDB.Connection (LanceDB v0.2.0)

Copy Markdown View Source

A connection to an embedded LanceDB database. Native resources are opaque.

Summary

Functions

Creates a shallow clone sharing source files. Options: source_version or source_tag. Retain the source files while clones reference them.

Connects to local or object storage, defaulting to "./lancedb". The second argument is a storage-options map passed to the SDK; the third is a keyword list of wrapper options

Creates an empty table with an explicit schema, accepting the same controls as create_table/4.

Creates a table from a row list or Source in one commit. Sources require a schema; row lists accept :schema or infer their types. Options apply to every input form

Drops a table from the database.

Opens a table, sharing the connection's session unless cache budgets are set.

Lists names with optional exclusive start_after and positive limit.

Types

connection()

@type connection() :: t()

result(value)

@type result(value) :: LanceDB.result(value)

t()

@opaque t()

table()

@type table() :: LanceDB.Table.t()

Functions

clone_table(conn, name, source_uri, opts \\ [])

@spec clone_table(connection(), String.t(), String.t(), keyword()) :: result(table())

Creates a shallow clone sharing source files. Options: source_version or source_tag. Retain the source files while clones reference them.

connect(uri \\ "./lancedb", storage_options \\ %{}, opts \\ [])

@spec connect(String.t(), map(), keyword()) :: result(connection())

Connects to local or object storage, defaulting to "./lancedb". The second argument is a storage-options map passed to the SDK; the third is a keyword list of wrapper options:

  • :read_consistency_interval_ms — nil (SDK default) or an integer from 0 through 4,294,967,295 milliseconds. Zero checks for external changes on every read; a positive interval checks on reads after that interval. The default does not automatically refresh after other handles write. Historical handles stay pinned. Use LanceDB.History.latest/1 to refresh.
  • :index_cache_size_bytes, :metadata_cache_size_bytes — integer byte counts from 0 through 9,223,372,036,854,775,807. Omit either option to use its SDK default; explicit nil is invalid. Either option configures the session on the initial SDK connection; zero disables the selected cache.
  • :timeout_ms — nil (no wrapper deadline) or a positive uint32 millisecond deadline for the full native connection. Expiry returns an error with LanceDB.Error.kind == :timeout.

Tables created or opened without per-open cache budgets share this session. Use separate connections for independent sessions. Per-open cache options select a separate session with SDK defaults for omitted budgets. Budgets limit session caches, not total native memory. Inspect actual usage with LanceDB.Table.cache_stats/1.

Native connection work runs on Tokio; the caller waits by message without retaining a BEAM scheduler. Caller death cancels the operation. Cancellation is cooperative: synchronous SDK work cannot be preempted, and separately spawned SDK tasks may outlive the operation.

create_empty_table(conn, name, schema, opts \\ [])

@spec create_empty_table(
  connection(),
  String.t(),
  LanceDB.Schema.t() | list(),
  keyword()
) ::
  result(table())

Creates an empty table with an explicit schema, accepting the same controls as create_table/4.

create_table(conn, name, input, opts \\ [])

@spec create_table(connection(), String.t(), [map()] | LanceDB.Source.t(), keyword()) ::
  result(table())

Creates a table from a row list or Source in one commit. Sources require a schema; row lists accept :schema or infer their types. Options apply to every input form:

  • :mode — :create (default), :overwrite, or :exist_ok. Existing tables returned by :exist_ok retain their settings and need not consume the source.
  • :schema_metadata — string metadata merged into the explicit schema.
  • :location — explicit table URI; pass it again when opening that location.
  • :storage_options — per-table storage overrides, merged with connection settings.
  • :max_rows_per_file, :max_rows_per_group, :max_bytes_per_file — positive uint32 writer limits. Byte limits are soft; omitted values use SDK defaults.
  • :data_storage_version — :stable, :next, or a version string: "legacy", "2.0", "2.1", "2.2", "2.3". Stable resolves to 2.1 in Lance 10.
  • :enable_stable_row_ids, :enable_v2_manifest_paths — explicit booleans; omitted settings inherit SDK/connection defaults. Blob schemas automatically require file format >= 2.2 and default to stable row IDs.
  • Blob write options — :external_blob_mode (:reference or :ingest), :allow_external_blob_outside_bases (default false, only for reference mode), and positive uint32 :blob_pack_file_size_threshold (SDK default 1 GiB).

Explicit typed format/row-ID/manifest options override corresponding storage options. Use file formats supported by every client that will open the table.

drop_table(conn, name)

@spec drop_table(connection(), String.t()) :: result(:dropped)

Drops a table from the database.

open_table(conn, name, opts \\ [])

@spec open_table(connection(), String.t(), keyword()) :: result(table())

Opens a table, sharing the connection's session unless cache budgets are set.

  • :branch — branch name; defaults to "main".
  • :version — positive version on that branch, pinning a read-only handle.
  • :location — explicit table URI, including custom creation locations.
  • :storage_options — a map merged over the connection's storage options.
  • :index_cache_size_bytes, :metadata_cache_size_bytes — the same byte ranges as connect/3. Either option creates a dedicated session. The omitted cache uses its SDK default, rather than the connection's budget. Zero disables that cache; explicit nil is invalid. The new session keeps the connection's storage and read-consistency configuration.
  • :timeout_ms — nil or a positive uint32 millisecond deadline covering the entire native open, including a dedicated session's connection setup.

Opening a table does not change other handles. Snapshots and branch checkouts retain the opened handle's session; branch creation uses a new SDK session. LanceDB.Table.cache_stats/1 reports the actual session's usage and counters. Expiry returns an error with LanceDB.Error.kind == :timeout. Native work runs on Tokio and cancels on caller death, with the cooperative limits of connect/3.

table_names(conn, opts \\ [])

@spec table_names(connection(), keyword()) :: result([String.t()])

Lists names with optional exclusive start_after and positive limit.