Reusable schemas contain Field structs, schema metadata, and tagged logical types.
Build one with Schema.new/2; LanceDB.Table.schema/1 returns the same representation.
Non-empty lists of field definitions are accepted as creation shorthand.
[id: :int64, text: :string, vector: {:vector, 3}]For nullability controls, use maps (which may be mixed with tuples):
[%{name: :id, type: :int64, nullable: false}, {:text, :string}]Field maps also accept :metadata, a map with string or atom keys and UTF-8
string values. This works at every struct depth. Table-level metadata is supplied
through the :schema_metadata creation option. LanceDB.Table.schema/1 includes
schema and field metadata, including empty maps.
Supported primitive types: :string, :binary, :boolean, :float16, :float32, :float64,
:int8, :int16, :int32, :int64, :uint8, :uint16, :uint32, :uint64,
:date32, and :date64. Other types are
{:vector, positive_dimension} (fixed-size float32 lists),
{:vector, positive_dimension, element_type} with :float16, :float32,
:float64, or :uint8,
{:timestamp, unit}, {:timestamp, unit, timezone}, {:list, type},
{:list, type, item_nullable: false}, and {:struct, fields}.
Lists and structs can nest; struct fields use the same definitions as table fields.
List items are nullable by default, independently of the list field's nullability.
Lists may be empty. Struct values are maps and must supply every child field,
including explicit nil values. Atom/string aliases must be unique at every level.
Lists and structs require explicit schemas; numeric-list inference remains vectors.
Lance 10 cannot store a null struct with required children, including through
nested struct fields. Supply a complete struct or make its child fields nullable.
Lance 10 also cannot cast struct update literals to required child fields;
use LanceDB.Table.upsert/3 or LanceDB.Table.merge/3 with complete records
for those columns. This also applies to field names that cannot round trip
through Arrow's SQL type parser, such as double-quoted names; literal updates
reject them before execution.
Literal updates replace whole top-level collections; nested-path updates are
not exposed. Schema evolution supports nested rename/drop/nullability changes.
Integer fields accept integers within their signed or unsigned storage range, without rounding or coercion. Binary fields accept ordinary Elixir binaries, including empty binaries and arbitrary non-UTF-8 bytes; results return binaries. String fields require valid UTF-8. These types also work inside lists and structs.
Blob fields use :blob and accept binaries, nil, or LanceDB.Blob.reference/1.
They also nest inside lists and structs. Byte fetching supports top-level and
struct-nested blobs; Lance 10 cannot address individual list elements by row ID.
Table queries return small descriptors;
retrieve bytes with LanceDB.Blob.fetch/3, LanceDB.Blob.fetch_ranges/3, or
LanceDB.Blob.open/3. A descriptor cannot be written back as a blob value.
The Arrow extension marker is managed internally and omitted from logical blob
field metadata. Blob types require explicit schemas and cannot be introduced
by casting another field's type. See LanceDB.Blob for streaming and cursor reads.
Date fields accept Calendar.ISO Date structs, YYYY-MM-DD strings, or epoch
integers: days for :date32, midnight milliseconds for :date64. Years must be
0001–9999; invalid dates and fractional days are rejected. Both return ISO date
strings. Dates, binary fields and additional integer widths require explicit
schemas; inference continues to produce strings and signed int64 integers.
Timestamp units are :second, :millisecond, :microsecond, and :nanosecond.
A timezone is nil (wall-clock time), "UTC", an IANA name such as
"Europe/Vienna", or a fixed offset such as "+02:00".
Timestamp values accept signed epoch integers in the field's unit, ISO 8601
strings, and Calendar.ISO Elixir date-time structs. Timezone-free fields use
NaiveDateTime or strings without offsets; fields with a timezone use
DateTime or strings with Z/an explicit offset. Subsecond precision must fit
the unit without rounding. Years are limited to 0001–9999 and the signed
64-bit storage range (about 1677–2262 for nanoseconds). Leap seconds are rejected.
Query results contain ISO strings: UTC with Z for timezone-bearing fields,
no offset for timezone-free fields. Nanoseconds remain exact in string results.
Timestamp types are never inferred from strings or date-time structs.
Additional explicit types:
{:decimal128, precision, scale}and{:decimal256, precision, scale}use exact plain decimal strings. Precision is 1–38 or 1–76; scale is -128 through precision. Floats, overflow and lossy fractions are rejected. Reads return strings. Negative-scale decimal literal updates require upsert or merge.{:time32, :second | :millisecond}and{:time64, :microsecond | :nanosecond}accept ISO time strings, ISOTimestructs, or integer ticks since midnight. Values must fit one day exactly. Reads return ISO strings.{:duration, unit}accepts and returns signed int64 ticks.{:fixed_size_binary, width}requires exactly that many bytes.:large_string,:large_binary, and{:large_list, item}use 64-bit offsets. Large list items accept the same nullability and Field forms as ordinary lists.{:dictionary, integer_key_type, scalar_value_type}accepts and returns decoded string, binary or numeric values, retaining dictionary encoding in Arrow. Key types are signed/unsigned integer widths.{:map, key_type, value_type}uses key/value pairs, returning pairs even for string keys. String/atom-keyed maps are also accepted as input. Keys are required and unique; values are nullable by default. Custom key/value Fields or{:map, entries_field}preserve nested properties. Maps require storage format 2.2 or newer, and sorted maps are unsupported. Use upsert or merge for whole-map literal updates. Collection types may nest.:jsonstores native Lance JSONB and accepts JSON values. SQL NULL is nil;LanceDB.JSON.null/0preserves a top-level JSON null. Reads return decoded values. SeeLanceDB.JSONfor the value contract. Arrow extension metadata is preserved in ArrowSchema and managed by logical JSON fields.
Binary vectors use uint8 elements: lists of integers from 0 through 255, with
dimensions measured in bytes. Their search/index distance must be :hamming.
Floating vector writes and queries reject values outside the element type's
finite range; float16/float32 storage rounds to that type's precision. Float64
queries preserve their precision. Batch search supports all four element types;
document multivector search remains float32 with the pinned upstream SDK.
Arrow batches may describe custom vector item fields using
{:vector, dimension, %LanceDB.Field{type: :float32}}. Table creation/evolution
reject nonstandard vector item names, nullability or metadata because Lance 10
cannot persist those properties. Unsupported external Arrow types are described
explicitly as {:unsupported, description} and cannot be silently recreated.
Fields are nullable by default. Every write must supply every field,
including explicit nil values for nulls. Vector elements cannot be null.
Summary
Functions
Builds a schema directly from logical field definitions. Invalid definitions raise ArgumentError.
Types
@type logical_type() :: primitive() | {:vector, pos_integer()} | {:vector, pos_integer(), primitive() | LanceDB.Field.t()} | {:timestamp, unit(), String.t() | nil} | {:list, LanceDB.Field.t()} | {:large_list, LanceDB.Field.t()} | {:fixed_size_binary, pos_integer()} | {:decimal128 | :decimal256, pos_integer(), integer()} | {:time32 | :time64 | :duration, unit()} | {:dictionary, primitive(), primitive()} | {:map, LanceDB.Field.t()} | {:struct, [LanceDB.Field.t()]} | {:unsupported, String.t()}
@type primitive() ::
:string
| :large_string
| :large_binary
| :json
| :binary
| :blob
| :boolean
| :float16
| :float32
| :float64
| :int8
| :int16
| :int32
| :int64
| :uint8
| :uint16
| :uint32
| :uint64
| :date32
| :date64
@type t() :: %LanceDB.Schema{fields: [LanceDB.Field.t()], metadata: map()}
@type unit() :: :second | :millisecond | :microsecond | :nanosecond