Xqlite (Xqlite v0.7.0)
View SourceThis is the central module of this library. All SQLite operations can be performed from here. Note that they delegate to other modules which you can also use directly.
Summary
Functions
Backs up a schema to a file.
Online backup with progress messages and cancellation. Accepts either a single cancel token or a list (OR-semantics).
Begins a transaction in the given mode (:deferred, :immediate, or
:exclusive). Emits [:xqlite, :transaction, :begin] telemetry.
Sets a plain sqlite3_busy_timeout on the connection, replacing any
xqlite-installed busy handler cleanly.
Signals a cancellation token. Emits [:xqlite, :cancel, :signalled].
Checks an existing table for values that would violate STRICT typing rules.
Commits the current transaction. Emits [:xqlite, :transaction, :commit].
Creates a cancellation token. Emits [:xqlite, :cancel, :token_created].
Deserializes a binary into a database, replacing its current contents.
Disables foreign key constraint enforcement for the given database connection (default behavior).
Disables strict mode only for the lifetime given database connection (SQLite's default).
Enables foreign key constraint enforcement for the given database connection.
Enables or disables extension loading on the connection.
Enables strict mode only for the lifetime of the given database connection.
Converts an existing table to STRICT mode via table rebuild.
Executes a non-returning SQL statement and returns a %Xqlite.Result{}.
Executes a SQL batch (multiple statements separated by semicolons).
Cancellable execute_batch/2. Accepts either a single cancel token or a list.
Cancellable execute/3. Accepts either a single cancel token or a list.
Runs a SQL statement and returns an %Xqlite.ExplainAnalyze{} report.
Reads a PRAGMA value from the connection.
Loads a SQLite extension from the shared library at path.
Opens a database connection with opinionated defaults and validated options.
Opens an in-memory database with opinionated defaults and validated options.
Opens a read-only connection to an in-memory SQLite database.
Executes a SQL query and returns a %Xqlite.Result{} struct.
Cancellable query/3. Accepts either a single cancel token or a list of
tokens; OR-semantics — any signalled token interrupts the query.
Cancellable query_with_changes/3. Accepts either a single cancel token or a list.
Registers a progress-tick subscriber on the connection.
Releases a savepoint. Emits [:xqlite, :savepoint, :release].
Removes any busy handler installed on the connection.
Restores a schema from a file.
Rolls back the current transaction. Emits
[:xqlite, :transaction, :rollback] with reason: :user_initiated.
Rolls back to a savepoint without releasing it. Emits
[:xqlite, :savepoint, :rollback_to].
Creates a savepoint with the given name. Emits
[:xqlite, :savepoint, :create].
Serializes a database to a contiguous binary.
Installs a busy handler on the connection.
Sets a PRAGMA value on the connection.
Creates a stream that executes a query and emits rows as string-keyed maps.
Unregisters a progress-tick subscriber by handle.
Performs a WAL checkpoint on the connection.
Types
@type conn() :: reference()
@type constraint_details() :: %{ message: String.t(), table: String.t() | nil, columns: [String.t()], index_name: String.t() | nil, constraint_name: String.t() | nil, source_type: storage_class(), target_type: storage_class() }
@type constraint_kind() ::
:constraint_check
| :constraint_commit_hook
| :constraint_datatype
| :constraint_foreign_key
| :constraint_function
| :constraint_not_null
| :constraint_pinned
| :constraint_primary_key
| :constraint_rowid
| :constraint_trigger
| :constraint_unique
| :constraint_vtab
| nil
@type error() :: {:error, error_reason()}
@type error_reason() :: :connection_closed | :execute_returned_results | :multiple_statements | :null_byte_in_string | :operation_cancelled | :unsupported_atom | {:cannot_convert_to_sqlite_value, String.t(), String.t()} | {:cannot_execute, String.t()} | {:cannot_execute_pragma, String.t(), String.t()} | {:cannot_open_database, String.t(), integer(), String.t()} | {:constraint_violation, constraint_kind(), constraint_details()} | {:database_busy_or_locked, String.t()} | {:expected_keyword_list, String.t()} | {:expected_keyword_tuple, String.t()} | {:expected_list, String.t()} | {:from_sql_conversion_failure, non_neg_integer(), atom(), String.t()} | {:index_exists, String.t()} | {:integral_value_out_of_range, non_neg_integer(), integer()} | {:internal_encoding_error, String.t()} | {:invalid_column_index, non_neg_integer()} | {:invalid_column_name, String.t()} | {:invalid_column_type, non_neg_integer(), String.t(), atom()} | {:invalid_parameter_count, %{provided: non_neg_integer(), expected: non_neg_integer()}} | {:invalid_parameter_name, String.t()} | {:invalid_pragma_name, String.t()} | {:invalid_stream_handle, String.t()} | {:lock_error, String.t()} | {:no_such_index, String.t()} | {:no_such_table, String.t()} | {:read_only_database, String.t()} | {:schema_changed, String.t()} | {:schema_parsing_error, String.t(), {:unexpected_value, String.t()}} | {:sql_input_error, sql_input_error()} | {:sqlite_failure, integer(), integer(), String.t() | nil} | {:table_exists, String.t()} | {:to_sql_conversion_failure, String.t()} | {:unsupported_data_type, atom()} | {:utf8_error, String.t()}
@type query_result() :: %{ columns: [String.t()], rows: [[sqlite_value()]], num_rows: non_neg_integer() }
@type storage_class() :: :integer | :real | :text | :blob | nil
Functions
Backs up a schema to a file.
Copies the named schema (default "main") to the file at dest_path. The
destination is created or overwritten. The source remains readable during
the backup.
@spec backup_with_progress( conn(), String.t(), String.t(), pid(), pos_integer(), reference() | [reference()] ) :: :ok | error()
Online backup with progress messages and cancellation. Accepts either a single cancel token or a list (OR-semantics).
Sends {:xqlite_backup_progress, remaining, pagecount} to pid after
each pages_per_step-page step. Returns {:error, :operation_cancelled}
if any token signals between steps.
Begins a transaction in the given mode (:deferred, :immediate, or
:exclusive). Emits [:xqlite, :transaction, :begin] telemetry.
@spec busy_timeout(conn(), non_neg_integer()) :: :ok | error()
Sets a plain sqlite3_busy_timeout on the connection, replacing any
xqlite-installed busy handler cleanly.
Calls remove_busy_handler/1 first (so our internal state is reclaimed
and {:xqlite_busy, …} messages stop for an understood reason), then
sets PRAGMA busy_timeout = ms.
Prefer this helper over reaching for PRAGMA busy_timeout directly: the
raw PRAGMA silently replaces any installed xqlite handler at the SQLite
level, stopping {:xqlite_busy, …} deliveries without clearing our
internal slot. This function keeps both sides consistent.
ms is the timeout in milliseconds. 0 disables the timeout entirely
(SQLite returns SQLITE_BUSY immediately on contention).
Signals a cancellation token. Emits [:xqlite, :cancel, :signalled].
Idempotent at the SQLite level — signalling twice is the same as once. Telemetry fires on every call, so consumers see distinct signal events even from repeated signals.
Checks an existing table for values that would violate STRICT typing rules.
Returns {:ok, []} if the table is clean, or {:ok, violations} where each
violation is a map with :rowid, :column, :actual_type, and :expected_type.
This is a read-only check — it does not modify the table.
Commits the current transaction. Emits [:xqlite, :transaction, :commit].
Creates a cancellation token. Emits [:xqlite, :cancel, :token_created].
The token is an opaque reference passed into cancellable operations
(query_cancellable/4, execute_cancellable/4, etc.). Signalling it via
cancel_operation/1 from any process interrupts in-flight cancellable
operations holding the same token.
Deserializes a binary into a database, replacing its current contents.
The binary must be a valid SQLite database image (as produced by
serialize/2). After deserialization the connection operates on the new
database entirely in memory.
schema identifies which attached database to replace (default "main").
read_only marks the deserialized image as read-only (default false).
Disables foreign key constraint enforcement for the given database connection (default behavior).
See enable_foreign_key_enforcement/1 for details.
Disables strict mode only for the lifetime given database connection (SQLite's default).
See enable_strict_mode/1 for details.
Enables foreign key constraint enforcement for the given database connection.
By default, SQLite parses foreign key constraints but does not enforce them. This function turns on enforcement.
Enables or disables extension loading on the connection.
Defaults to true. Wraps XqliteNIF.enable_load_extension/2 and emits
[:xqlite, :extension, :enable] telemetry.
Enables strict mode only for the lifetime of the given database connection.
In strict mode, SQLite is less forgiving. For example, an attempt to insert
a string into an INTEGER column of a STRICT table will result in an error,
whereas in normal mode it might be coerced or stored as text.
This setting only affects tables declared with the STRICT keyword.
See: STRICT Tables
Converts an existing table to STRICT mode via table rebuild.
This creates a new STRICT table, copies all data, drops the original, and renames the new table — all inside a transaction.
If existing data violates STRICT typing rules, the operation fails with
{:error, {:strict_violations, violations}} where violations is a list
of maps from check_strict_violations/2. The original table is left untouched.
Options
None currently.
Examples
:ok = Xqlite.enable_strict_table(conn, "users")
Executes a non-returning SQL statement and returns a %Xqlite.Result{}.
For DML statements, changes contains the number of affected rows.
Executes a SQL batch (multiple statements separated by semicolons).
Wraps XqliteNIF.execute_batch/2 and emits [:xqlite, :execute_batch, :*]
telemetry. No parameter binding inside the batch.
Cancellable execute_batch/2. Accepts either a single cancel token or a list.
@spec execute_cancellable( conn(), String.t(), list(), reference() | [reference()] ) :: {:ok, non_neg_integer()} | error()
Cancellable execute/3. Accepts either a single cancel token or a list.
@spec explain_analyze(conn(), String.t(), list() | keyword()) :: {:ok, Xqlite.ExplainAnalyze.t()} | error()
Runs a SQL statement and returns an %Xqlite.ExplainAnalyze{} report.
The statement is executed in full (rows are fetched and discarded). The
returned struct combines the static EXPLAIN QUERY PLAN tree with
runtime counters from sqlite3_stmt_scanstatus_v2 / sqlite3_stmt_status
and a wall-clock measurement around the execution. See
Xqlite.ExplainAnalyze for the field layout and how to interpret it.
Examples
iex> {:ok, conn} = Xqlite.open_in_memory()
iex> XqliteNIF.execute_batch(conn, "CREATE TABLE t(id INTEGER PRIMARY KEY, name TEXT); INSERT INTO t(name) VALUES ('a'), ('b');")
:ok
iex> {:ok, report} = Xqlite.explain_analyze(conn, "SELECT name FROM t WHERE name = ?", ["b"])
iex> match?(%Xqlite.ExplainAnalyze{}, report)
true
Reads a PRAGMA value from the connection.
Wraps XqliteNIF.get_pragma/2 and emits [:xqlite, :pragma, :get].
Loads a SQLite extension from the shared library at path.
entry_point is the extension's init function name; pass nil (default)
to let SQLite auto-detect. Extension loading must be enabled first via
enable_load_extension/2.
Opens a database connection with opinionated defaults and validated options.
All PRAGMAs are applied on the same connection immediately after opening, with no window for another process to observe an unconfigured state.
Options
:journal_mode- SQLite journal mode.:walenables concurrent readers with a single writer. The default value is:wal.:busy_timeout(timeout/0) - Milliseconds to wait when the database is locked.:infinitywaits forever. The default value is5000.:foreign_keys(boolean/0) - Enable foreign key constraint enforcement. SQLite defaults to OFF. The default value istrue.:synchronous- Synchronous mode.:normalis safe with WAL and significantly faster than:full. The default value is:normal.:cache_size(integer/0) - Page cache size. Negative values mean KB (e.g.,-64000= 64MB). SQLite default is 2MB. The default value is-64000.:temp_store- Where to store temporary tables and indices. The default value is:memory.:wal_autocheckpoint(non_neg_integer/0) - WAL auto-checkpoint threshold in pages. 0 disables auto-checkpoint. The default value is1000.:mmap_size(non_neg_integer/0) - Memory-mapped I/O size in bytes. 0 disables mmap. The default value is0.:auto_vacuum- Auto-vacuum mode. Must be set before creating any tables. The default value is:none.
Examples
{:ok, conn} = Xqlite.open("my.db")
{:ok, conn} = Xqlite.open("my.db", journal_mode: :delete, busy_timeout: 10_000)
Opens an in-memory database with opinionated defaults and validated options.
Accepts the same options as open/2.
Opens a read-only connection to an in-memory SQLite database.
Useful for connecting to a named shared-cache in-memory database opened
read-write by another connection — pass its URI as uri, or omit it to
open a private (empty) read-only :memory: database.
No PRAGMAs are applied; read-only databases can't persist most settings.
Executes a SQL query and returns a %Xqlite.Result{} struct.
For SELECT queries, num_rows is the count of returned rows and changes
is 0. For DML (INSERT/UPDATE/DELETE), num_rows is 0 (no result rows)
and changes is the number of affected rows.
Uses XqliteNIF.query_with_changes/3 which captures the affected row count
atomically inside the connection lock. For zero-overhead access without the
changes field, use XqliteNIF.query/3 directly.
@spec query_cancellable( conn(), String.t(), list() | keyword(), reference() | [reference()] ) :: {:ok, query_result()} | error()
Cancellable query/3. Accepts either a single cancel token or a list of
tokens; OR-semantics — any signalled token interrupts the query.
See XqliteNIF.query_cancellable/4 for the raw NIF (list form only).
@spec query_with_changes_cancellable( conn(), String.t(), list() | keyword(), reference() | [reference()] ) :: {:ok, map()} | error()
Cancellable query_with_changes/3. Accepts either a single cancel token or a list.
@spec register_progress_hook(conn(), pid(), keyword()) :: {:ok, non_neg_integer()} | error()
Registers a progress-tick subscriber on the connection.
After every ~64 SQLite VM instructions × every_n, sends
{:xqlite_progress, count, elapsed_ms} # tag = nil
{:xqlite_progress, tag, count, elapsed_ms} # tag setto pid. count is the per-subscriber decimated counter; elapsed_ms
is the wall time since this specific subscriber was registered.
Multiple subscribers can coexist independently — each gets its own opaque handle, and unregistering one never affects another.
Options
:every_n(positive integer, default1000) — emit every Nth progress callback fire. The progress callback fires every 8 SQLite VM instructions (currently fixed);every_ndecimates further.:tag(atom, defaultnil) — included in each emitted message as the second tuple element when set. Useful when a single listener process subscribes to multiple connections and needs to tell them apart without spawning a process per connection.
Returns {:ok, handle} where handle is the value to pass to
unregister_progress_hook/2. Returns {:error, reason} on failure.
Releases a savepoint. Emits [:xqlite, :savepoint, :release].
Removes any busy handler installed on the connection.
Safe to call when no handler is installed (no-op on both the SQLite and
xqlite sides). After removal, SQLite returns SQLITE_BUSY immediately on
contention unless a busy_timeout is subsequently set (see
busy_timeout/2).
Restores a schema from a file.
Replaces the named schema (default "main") with the contents of the file
at src_path. Existing data in that schema is overwritten.
Rolls back the current transaction. Emits
[:xqlite, :transaction, :rollback] with reason: :user_initiated.
SQLite-internal rollbacks (constraint violations, deferred-FK failures
at commit time) surface as errors from commit/1 rather than passing
through here — those events come from the register_rollback_hook/2
fan-out instead.
Rolls back to a savepoint without releasing it. Emits
[:xqlite, :savepoint, :rollback_to].
Note: this does NOT invoke SQLite's rollback_hook — that fires only
for outer-transaction rollbacks. Use register_rollback_hook/2 for
outer rollback observability; this telemetry event is what's
available for partial-rollback observability.
Creates a savepoint with the given name. Emits
[:xqlite, :savepoint, :create].
Serializes a database to a contiguous binary.
Returns a binary snapshot of the entire database — an atomic, point-in-time copy. No pages are locked during serialization.
schema identifies which attached database to serialize. Defaults to
"main". Use "temp" for the temp database or the name of an attached
database.
Installs a busy handler on the connection.
When SQLite encounters a locked database (another writer holds
RESERVED+) the handler decides whether to retry or surface
SQLITE_BUSY to the caller. Each invocation is also forwarded to pid as
{:xqlite_busy, retries_so_far, elapsed_ms}so callers can observe contention (telemetry, structured logging, adaptive backoff).
Options
:max_retries(non-negative integer, default50) — stop after this many retries and let the caller seeSQLITE_BUSY.:max_elapsed_ms(non-negative integer, default5_000) — absolute time ceiling in milliseconds from the first busy event in the window.:sleep_ms(non-negative integer, default10) — milliseconds to sleep between retries. Zero disables the pause (tight spin; rarely what you want).
Replacing an existing handler is atomic: the previous handler's state is reclaimed before the new one takes effect.
Warning — PRAGMA busy_timeout silently replaces your handler
SQLite allows the busy handler to be silently replaced by
sqlite3_busy_timeout, PRAGMA busy_timeout, or another
sqlite3_busy_handler call. If you install an xqlite handler and then
run PRAGMA busy_timeout = N (or XqliteNIF.set_pragma(conn, "busy_timeout", ms)), SQLite replaces our callback with its built-in
sleep-and-retry one and you stop receiving {:xqlite_busy, …}
messages. No memory is leaked — our internal state is reclaimed on the
next set_busy_handler/3, remove_busy_handler/1, or connection
close — but the messages will have silently stopped.
To switch to plain-timeout semantics without surprises, use
busy_timeout/2.
Sets a PRAGMA value on the connection.
Wraps XqliteNIF.set_pragma/3 and emits [:xqlite, :pragma, :set].
Creates a stream that executes a query and emits rows as string-keyed maps.
This provides a high-level, idiomatic Elixir Stream for processing large
result sets without loading them all into memory at once. Rows are fetched
from the database in batches as the stream is consumed.
Options
:batch_size(integer, default:500) - The maximum number of rows to fetch from the database in a single batch.:type_extensions(list of modules, default:[]) - A list of modules implementing theXqlite.TypeExtensionbehaviour. Parameters are encoded before binding, and result values are decoded as rows are fetched. Extensions are applied in list order; the first match wins.
Examples
iex> {:ok, conn} = Xqlite.open_in_memory()
iex> XqliteNIF.execute_batch(conn, "CREATE TABLE users(id, name); INSERT INTO users VALUES (1, 'Alice'), (2, 'Bob');")
:ok
iex> Xqlite.stream(conn, "SELECT id, name FROM users;") |> Enum.to_list()
[%{"id" => 1, "name" => "Alice"}, %{"id" => 2, "name" => "Bob"}]Returns an Enumerable.t() on success or {:error, reason} on setup failure.
Callers must pattern-match the result before piping — this is intentional,
as returning a stream that silently errors on first consume would hide
setup failures (e.g., invalid SQL, closed connection).
Errors that occur during stream consumption (e.g., database connection lost mid-stream) will be logged and will cause the stream to halt.
@spec unregister_progress_hook(conn(), non_neg_integer()) :: :ok | error()
Unregisters a progress-tick subscriber by handle.
Idempotent — unregistering an unknown handle returns :ok. Returns
{:error, :connection_closed} if the connection is closed.
Performs a WAL checkpoint on the connection.
mode is one of :passive (default), :full, :restart, or :truncate.
schema is the attached-database name (default "main").
Returns {:ok, %{log_pages, checkpointed_pages, busy?}} on success.