Development and tests

Back to README.

Run all commands from the package directory.

Development

Use rustup so rust-toolchain.toml selects Rust 1.99.0, rustfmt and Clippy. Run these checks from the package directory. Set the source-build flag before you get dependencies:

export LANCEDB_BUILD=true
mix deps.get
mix format --check-formatted
mix credo --strict
mix compile --warnings-as-errors
mix test --cover
cargo fmt --all --check
cargo clippy --locked --release --all-targets -p lancedb_nif -- -D warnings
cargo test --locked --release -p lancedb_nif

Each test directory has a specific purpose:

Without LANCEDB_BUILD=true, tests use the published NIF. Set LANCEDB_BUILD=true before dependency resolution to check changes to Rust. The default suite does not need an S3 service or Phoenix application. CI runs the S3 suite separately.

Integration tests check persistence, scalar and recursive schemas, searches, indexes, writes, ingestion, and schema evolution. They also check history, branches, metadata, blobs, maintenance, telemetry, resource ownership, and failure contracts. LanceDB.DataCase allocates a unique path per test and cleans it on exit. Tests that change process-global admission limits, telemetry recorder state or environment variables run synchronously.

StreamData is a test-only dependency. Properties check recursive JSON and typed values, exact integers and decimals, float32 rounding, schema metadata, expression quoting, query composition and overflow limits. Independent models check ordered pagination and nested sorting (including null ordering), upsert and conditional merge sequences, metadata updates and branch isolation. Vector properties verify both candidate completeness and distances against arithmetic oracles.

Write-failure properties put incorrect rows, producer failures, and finalizer failures at generated batch boundaries. They check stored rows, versions, confirmed commits, and source cleanup. They also check the difference between commit: :once and commit: :batch. If a finalizer fails during early termination, the result includes Error.details.cleanup_error. The result keeps the primary error and confirmed progress.

Native safety tests reject excessively nested malformed records, including a 10,000-level input tested in a separate BEAM. Term decoding limits wire nesting to 64 levels, including the outer row list and row map. Cursor and job scheduler tests run a separate BEAM with one dirty I/O scheduler and hold storage responses in a local read-only S3 fixture. Pending cursor reads, cleanup, and native job waits use Tokio and reply by message. BEAM schedulers stay available for other work. Storage scheduler tests use one dirty CPU scheduler and verify that held history, branch, table and index metadata I/O, prewarming, listing, and deletion do not delay Arrow conversion. The table URI accessor is checked to complete without a storage request. Table-cache regressions gate background workers and S3 index reads to check coalescing, bounded concurrency, caller cleanup, LRU/idle eviction, retry, and invalidation during active work. They check index replacement and appended segments, shared session counters, and fewer first-query S3 index reads after warming than with an independent cold session. Connect/open scheduler tests hold both S3 reads and local SDK region-resolution requests. They check deadlines and caller death with one dirty CPU and one dirty I/O scheduler, including admission release before storage is released. Source-build regressions use tiny Cargo workspaces and run only with LANCEDB_BUILD=true. Properties that change data create a fresh database inside check all with with_database/1.

This helper removes the database after each generation and shrink attempt. Only properties that read data share a fixture. Generators limit recursion and dataset size so failures shrink quickly. They include empty values, nulls, integer boundaries, Unicode, arbitrary bytes, and quoting characters. When you correct a bug, add its useful counterexample as a regression test.

Use the StreamData :property tag to select all properties, including properties in feature suites. A filename pattern can miss those properties. To repeat a failure, use the seed that ExUnit prints:

mix test test/unit
mix test --only property
mix test --only property --seed 12345
mix test test/properties/lancedb_write_failure_test.exs --seed 12345
scripts/test_s3.sh

CI runs formatting, strict Elixir linting, Clippy with warnings denied for all native targets, locked Cargo tests and the full Elixir suite with coverage. Before resolving dependencies or compiling Rust, it compares committed native inputs against the source revision selected in native-build.exs. Native source, Cargo files, toolchain, or precompile workflow changes require a newly published and verified native build with matching build and checksum manifests. Elixir-only changes can retain the selected build. Run the same check locally with elixir scripts/check_native_build.exs after committing changes; it compares the selected revision with HEAD and requires that revision in local Git history. It does not download or verify archives. CI also replays every property with seeds 0 and 12345, in addition to the full suite's random seed. Elixir line coverage must be at least 90%. Coverage excludes test support and the internal NIF module, whose fallback stubs are replaced by Rust NIFs. Elixir coverage does not measure Rust coverage.

See Internal responsibilities and SDK compatibility for development notes.

S3 integration tests

GitHub CI has a separate MinIO integration job. It covers writes, independent connections, vector, FTS, and hybrid search, version tags, restore, cleanup, table deletion, and rejected credentials. Run the same job locally with Elixir and curl. Curl must support AWS SigV4:

scripts/test_s3.sh

The script downloads the pinned MinIO release from its official GitHub assets and verifies it against a committed SHA256 digest. It supports the same two Linux platforms as the NIF and caches the verified binary under _build/minio. The server binds to localhost on an ephemeral port, uses temporary data and test credentials, and is stopped on exit. Failed runs print its logs before cleanup. Docker, an AWS account, and GitHub secrets are not necessary.

S3 tests are tagged :s3. Ordinary mix test runs exclude them. To use an existing test endpoint and bucket, set these environment variables:

• LANCEDB_TEST_S3_ENDPOINT
• LANCEDB_TEST_S3_URI (an S3 bucket or prefix)
• LANCEDB_TEST_S3_ACCESS_KEY
• LANCEDB_TEST_S3_SECRET_KEY

Then run:

LANCEDB_BUILD=true mix test test/lancedb_s3_test.exs --include s3

Each test creates and removes tables under a unique prefix beneath the supplied URI. These tests check S3 compatibility with MinIO. They do not cover AWS IAM, STS, or AWS-specific service behavior.

Connection and open benchmark

The benchmark compares per-open cache budgets with configuring those budgets on the initial connection. It reports connect and open separately, plus the combined per-sample time, as JSON with median and p95 values in microseconds. Run local storage alone, or reuse the disposable MinIO server for both:

MIX_ENV=test LANCEDB_BUILD=true mix run benchmarks/connection_open.exs
MIX_ENV=test LANCEDB_BUILD=true scripts/test_s3.sh mix run benchmarks/connection_open.exs

Set LANCEDB_BENCH_ITERATIONS to change the default 100 samples per case. --legacy-only runs against a checkout before connection budgets were added. Five untimed opens prime SDK initialization and module loading. Cold cases create a fresh connection/session for each sample; they do not flush OS or storage caches. Warm cases retain a connection after one FTS index prewarm. The existing per-open budget path continues to create a new session on each open, whereas the configured connection reuses its session. The workload has 100 rows, a text index, 112 MiB index and 16 MiB metadata budgets, and a zero read-consistency interval. Queries and index prewarming are outside the timed open samples. Setup time for each warm case is reported separately.

On 2026-10-11, Linux x86_64 / Ryzen 7 8845HS, Rust 1.99.0 release builds, Elixir 1.20.4 / OTP 29, and localhost MinIO RELEASE.2025-09-07T16-13-09Z:

Combined medians are computed per sample, so they need not equal the sum of the phase medians. The paired comparison shows about 48% lower cold combined time and 96% lower warm open time on MinIO. This sequential microbenchmark does not predict production S3 latency or FTS query performance; host load, SDK cache reuse, and namespace setup affect its results. The before-patch baseline was captured on commit 1f45a1f; the comparison runs both paths in the patched build. Recorded phase summaries are in benchmarks/results/connection_open_2026-10-11.json.