Sqlcx (sqlcx v2.0.0-rc.1) View Source
Sqlcx gives you a way to create and query SQLCipher (encrypted SQLite) databases.
Basic Example
iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.exec(db, "CREATE TABLE t (a INTEGER, b INTEGER, c INTEGER)")
:ok
iex> Sqlcx.exec(db, "INSERT INTO t VALUES (1, 2, 3), (4, 5, 6)")
:ok
iex> Sqlcx.query(db, "SELECT * FROM t")
{:ok, [[1, 2, 3], [4, 5, 6]]}
iex> Sqlcx.query_rows(db, "SELECT * FROM t")
{:ok, [%{a: 1, b: 2, c: 3}, %{a: 4, b: 5, c: 6}]}
Configuration
Sqlcx uses the Erlang library esqlcipher
which accepts a timeout parameter for almost all interactions with the database.
The default value for this timeout is 5000 ms. Many functions in Sqlcx accept
a :db_timeout
option that is passed on to the esqlcipher calls and also defaults
to 5000 ms. If required, this default value can be overridden globally with the
following in your config.exs
:
config :sqlcx, db_timeout: 10_000 # or other positive integer number of ms
Another esqlcipher parameter is :db_chunk_size
. This is the number of rows
that are read from native sqlcipher and send to the Erlang process in one bulk.
For example, consider a table mytable
that has 1000 rows. We make the query
to get all rows with db_chunk_size: 500
parameter:
Sqlcx.query(db, "select * from mytable", db_chunk_size: 500)
In this case, all rows will be passed from native sqlcipher OS thread to the
Erlang process in two passes with each pass containing 500 rows. This reduces
the overhead of transmitting and copying rows from the native OS sqlcipher
thread to the Erlang process by chunking the list of result rows. There will
be no difference visible to the end user: The result of the Sqlcx.query
call
will be the same no matter the chunk size.
Please decrease :db_chunk_size
if rows are heavy. The default value is 5000
.
If you’re in doubt what to do with this parameter, just leave it as is.
The default value will be ok in almost all cases. You can also change the
default value like this:
config :sqlcx, db_chunk_size: 500 # if most of the database rows are heavy
Link to this section Summary
Types
Database connection to an encrypted or unencrypted database.
Database connection to an encrypted database., subtype of connection()
An error that can be returned by sqlcipher calls. Contains an error type atom and a reason string.
Database password
Database connection to an unencrypted database, subtype of connection()
Options that can be passed to the query
functions.
Functions
Return the number of the rows that have been modified, inserted, or deleted by the last statement.
Close the database connection.
Send a raw SQL statement to the database.
Same as exec/3
but raises a Sqlcx.QueryError on error. Returns :ok
otherwise.
Whether conn
is a database connection. Can be used in guards.
Whether conn
is a connection to an encrypted database. Can be used in guards.
Whether conn
is a connection to an unencrypted/plaintext database. Can be used in guards.
Open an sqlite3/sqlcipher database connection that can be queried. If the database file does not yet exist, it will be created.
Shortcut for open/2
, raises errors instead of returning error tuples.
Runs a query and returns the results.
Same as query/3
but raises a Sqlcx.QueryError
on error.
Runs a query and returns the results as a list of rows each represented as a map.
Same as query_rows/3
but raises a Sqlcx.QueryError on error.
Change the password of an encrypted database.
Shortcut for rekey/3
, raises errors instead of returning error tuples.
Run a code block inside a SAVEPOINT
transaction. Unlike regular
BEGIN
/COMMIT
transactions, savepoints can be nested.
Sets a PID to receive notifications about table updates.
Run a code block inside a transaction. If the do
block does not raise an exception,
the transaction will be commited via COMMIT
. Otherwise, ROLLBACK
will be called.
Opens a database connection, executes fun
on it and closes it.
Opens a database connection, executes fun
on it and closes it.
Runs fun
inside a SAVEPOINT
transaction. Unlike regular BEGIN
/COMMIT
transactions, savepoints can be nested.
Runs fun
inside a transaction. If fun
returns without raising an exception,
the transaction will be commited via COMMIT
. Otherwise, ROLLBACK
will be called.
Link to this section Types
Specs
connection()
Database connection to an encrypted or unencrypted database.
Specs
encrypted_connection()
Database connection to an encrypted database., subtype of connection()
Specs
An error that can be returned by sqlcipher calls. Contains an error type atom and a reason string.
Specs
open_encrypted_options() :: [db_password: password(), db_timeout: timeout()] | [db_timeout: timeout(), db_password: password()] | [{:db_password, password()}]
Options that can be passed to open/2
or open!/2
for encrypted databases.
Specs
Options that can be passed to open/2
, open!/2
, or with_db/3
.
Specs
open_plaintext_options() :: [{:db_timeout, timeout()}]
Options that can be passed to open/2
or open!/2
for unencrypted databases.
Specs
Database password
Specs
plaintext_connection()
Database connection to an unencrypted database, subtype of connection()
Specs
query_options() :: [ bind: [Sqlcx.Statement.bind_value()], into: Collectable.t(), db_timeout: timeout(), db_chunk_size: pos_integer() ]
Options that can be passed to the query
functions.
Link to this section Functions
Specs
changes(connection(), [{:db_timeout, timeout()}]) :: non_neg_integer()
Return the number of the rows that have been modified, inserted, or deleted by the last statement.
See https://www.sqlite.org/c3ref/changes.html for details information.
Example
iex> {:ok, db} = Sqlcx.open(":memory:") iex> Sqlcx.exec(db, "create table test (val integer)") :ok iex> Sqlcx.exec(db, "insert into test(val) values (1), (2)") :ok iex> Sqlcx.changes(db) 2
Specs
close(connection(), [{:db_timeout, timeout()}]) :: :ok | error()
Close the database connection.
Specs
exec(connection(), String.t(), [{:db_timeout, timeout()}]) :: :ok | error()
Send a raw SQL statement to the database.
This function is intended for running simple/one-off SQL statements.
Parameters are not supported here, see query/3
related functions for those!
No results are returned; queries return either :ok
or an error.
This is generally useful for things like re-playing a SQL export back into the
database or for simple commands like BEGIN
or COMMIT
(but see also
with_transaction/3
and with_savepoint/4
for transactions).
Do not manually insert values into SQL statements as that is prone to
SQL injection.
If you need the results returned by a query, use the Sqlcx.Query
interface
(see also the query/3
and query_rows/3
shortcuts) or work directly with
prepared statements using Sqlcx.Statement
.
Specs
exec!(connection(), String.t(), [{:db_timeout, timeout()}]) :: :ok
Same as exec/3
but raises a Sqlcx.QueryError on error. Returns :ok
otherwise.
Whether conn
is a database connection. Can be used in guards.
Whether conn
is a connection to an encrypted database. Can be used in guards.
Whether conn
is a connection to an unencrypted/plaintext database. Can be used in guards.
Specs
open(String.t(), open_plaintext_options()) :: {:ok, plaintext_connection()} | error()
open(String.t(), open_encrypted_options()) :: {:ok, encrypted_connection()} | error()
Open an sqlite3/sqlcipher database connection that can be queried. If the database file does not yet exist, it will be created.
Pass [db_password: "1234"]
as an options to open an sqlcipher encrypted
database. If the database does not exist, an encrypted database will be
created on first write. The actual encryption key will be derived from
db_password
using PBKDF2 key derivation by sqlcipher. However, it's possible
to specify a raw byte sequence as a key using [db_password: {:raw, <<...>>}]
.
The raw key has to be either 32 or 48 bytes long where the key is 32 bytes and
the optional salt is another 16 bytes long. If the salt is not explicitly
provided, it will be generated randomly and stored in the first 16 bytes of
the database. The raw binary key will be internally hex-encoded and passed to
sqlcipher as x'A0B1C2(...)D3E4F5'
with either 64 or 96 hex chaarcters. In
order to avoid confusion or unsafe behavior, string passwords of that format
will return an {:error, {:badarg, _}}
.
Please refer to the sqlcipher documentation
for further information about the generation and usage of encryption keys.
Note that opening encrypted databases can take quite some time. This is by
design because key derivation uses the PBKDF2 algorithm. If on a slower
machine, adjust the timeout accordingly.
Returns {:ok, db}
where db
is a database connection if the file could be
opened or created and could be read (i.e., the db_password
was correct or
the database is not encrypted).
If the password is incorrect or a password was given on an unencrypted database
(or vice versa), or if the file is corrupted or unreadable, will return an
error of the form {:error, {:baddb, _}}
.
Remember to close/2
the database connection when it is no longer needed
(although it will automatically be closed when its process dies or it is
garbage collected)
Specs
open!(String.t(), open_plaintext_options()) :: plaintext_connection()
open!(String.t(), open_encrypted_options()) :: encrypted_connection()
Shortcut for open/2
, raises errors instead of returning error tuples.
Specs
query(connection(), String.t(), query_options()) :: {:ok, [keyword()]} | error()
Runs a query and returns the results.
Parameters
db
- An SQLCipher database connection.sql
- The query to run as a string.opts
- Options to pass into the query. See below for details.
Options
bind
- List/keyword list of values to bind to the sqlite parameters. SeeStatement.bind/3
for details.into
- The collection to put each row into. If provided, values arezip
ed with the column names, sointo: %{}
produces a Map (note that column names are Strings, not atoms). Defaults tonil
which simply outputs each row as a list of values without any column names.db_timeout
- The timeout (in ms) to apply to each of the underlying SQLCipher operations. Defaults toApplication.get_env(:sqlcx, :db_timeout)
or5000
ms if not configured.db_chunk_size
- The number of rows to read from native sqlite and send to erlang process in one bulk. Defaults toApplication.get_env(:sqlcx, :db_chunk_size)
or5000
if not configured.
Returns
{:ok, [[values...]...]}
on success{:error, _}
on failure.
Example
iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.exec(db, "CREATE TABLE data (cat, name)")
:ok
iex> Sqlcx.exec(db, "INSERT INTO data VALUES (1, 'a'), (2, 'b'), (2, 'c')")
:ok
iex> Sqlcx.query(db, "SELECT * FROM data WHERE cat = :cat", bind: [cat: 2])
{:ok, [[2, "b"], [2, "c"]]}
iex> Sqlcx.query(db, "SELECT * FROM data WHERE cat = :cat", bind: [cat: 2], into: [])
{:ok, [[{"cat", 2}, {"name", "b"}], [{"cat", 2}, {"name", "c"}]]}
iex> Sqlcx.query(db, "SELECT * FROM data WHERE cat = :cat", bind: [cat: 2], into: %{})
{:ok, [%{"cat" => 2, "name" => "b"}, %{"cat" => 2, "name" => "c"}]}
iex> Sqlcx.close(db)
:ok
Specs
query!(connection(), String.t(), query_options()) :: [Enum.t()]
Same as query/3
but raises a Sqlcx.QueryError
on error.
Returns the results otherwise.
Specs
query_rows(connection(), String.t(), query_options()) :: {:ok, %{}} | error()
Runs a query and returns the results as a list of rows each represented as a map.
Unlike query/3
, column names are translated into atoms and rows are Map
s
so values can be accessed idiomatically using a row.value
syntax.
Questioning: This function might be dropped in future versions.
Parameters
db
- An SQLCipher database.sql
- The query to run as a string.opts
- Options to pass into the query. See below for details.
Options
bind
- List/keyword list of values to bind to the sqlite parameters. SeeStatement.bind/3
for details.into
- The collection to put each row into. Use[]
to get keyword lists. Defaults to%{}
.db_timeout
- The timeout (in ms) to apply to each of the underlying SQLCipher operations. Defaults toApplication.get_env(:sqlcx, :db_timeout)
or5000
ms if not configured.db_chunk_size
- The number of rows to read from native sqlite and send to erlang process in one bulk. Defaults toApplication.get_env(:sqlcx, :db_chunk_size)
or5000
if not configured.
Returns
{:ok, [%{name: value}]}
on success{:error, _}
on failure.
Example
iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.exec(db, "CREATE TABLE data (cat, name)")
:ok
iex> Sqlcx.exec(db, "INSERT INTO data VALUES (1, 'a'), (2, 'b'), (2, 'c')")
:ok
iex> Sqlcx.query_rows(db, "SELECT * FROM data WHERE cat = :cat", bind: [cat: 2])
{:ok, [%{cat: 2, name: "b"}, %{cat: 2, name: "c"}]}
iex> Sqlcx.query_rows(db, "SELECT * FROM data WHERE cat = :cat", bind: [cat: 2], into: [])
{:ok, [[cat: 2, name: "b"], [cat: 2, name: "c"]]}
iex> Sqlcx.close(db)
:ok
Specs
query_rows!(connection(), String.t(), query_options()) :: %{}
Same as query_rows/3
but raises a Sqlcx.QueryError on error.
Returns the results otherwise.
Specs
rekey(encrypted_connection(), String.t(), [{:db_timeout, timeout()}]) :: :ok | error()
Change the password of an encrypted database.
password
can either be a String or a {:raw, <<...>>}
tuple containing a
binary raw key. See open/2
.
Note that this function cannot be used to encrypt an unencrypted database.
Returns :ok
or an error/0
if the new key is invalid or rekeying the
database fails.
Specs
rekey!(encrypted_connection(), String.t(), [{:db_timeout, timeout()}]) :: :ok
Shortcut for rekey/3
, raises errors instead of returning error tuples.
Run a code block inside a SAVEPOINT
transaction. Unlike regular
BEGIN
/COMMIT
transactions, savepoints can be nested.
If the do
block returns without raising an exception, the savepoint will be
commited via RELEASE SAVEPOINT
. Otherwise, ROLLBACK TO SAVEPOINT
will be called.
Even if an inner savepoint is committed, it may be rolled back if an outer
transaction is rolled back.
Since this is implemented as a macro, Sqlcx
must be require
d or import
ed
in order to use it.
See also the with_savepoint/3
version which takes a function as an argument.
Note that this version does not take any further arguments. Thus, the savepoint
will be automatically given a random name and the default timeout will be
applied to the SAVEPOINT
, RELEASE
and ROLLBACK
commands.
Returns {:ok, result}
if the function succeeds, {:exception, err}
if the
function raises an exception, {:error, err}
if any of the savepoint sqlite
commands fails.
Examples
iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.savepoint db do
...> Sqlcx.exec(db, "create table foo(id integer)")
...> Sqlcx.exec(db, "insert into foo (id) values(42)")
...> try do
...> Sqlcx.savepoint db do
...> Sqlcx.exec(db, "insert into foo (id) values (3)")
...> :ok = Sqlcx.exec(db, "not a valid sql query") # Raises an exception
...> end # will be rolled back because the do block fails
...> rescue
...> _ -> "we don't care about the exception"
...> end
...> end
iex> Sqlcx.query_rows(db, "select * from foo")
{:ok, [%{id: 42}]}
Specs
set_update_hook(connection(), pid(), [{:db_timeout, timeout()}]) :: :ok | error()
Sets a PID to receive notifications about table updates.
Messages will come in the shape of:
{action, table, rowid}
action
: either:insert
,:update
, or:delete
table
: charlist of the table name. Example:'posts'
rowid
: internal rowid index of the row. If the table contains anINTEGER PRIMARY KEY
column, this column will be the row id. Do note thatINT PRIMARY KEY
and similar will not be an alias for the row id.
Also note that update hooks do not work on
WITHOUT ROWID
tables.
See the sqlite3 docs for update_hook and for row ids.
Run a code block inside a transaction. If the do
block does not raise an exception,
the transaction will be commited via COMMIT
. Otherwise, ROLLBACK
will be called.
Note that transactions cannot be nested. If you need nested transactions,
use savepoint/2
.
Since this is implemented as a macro, Sqlcx
must be require
d or import
ed
in order to use it.
See also the with_transaction/3
version which takes a function as an argument.
Note that this version does not take any further arguments. The default
timeout will be applied to the BEGIN
, COMMIT
and ROLLBACK
commands.
Returns the result of the function if it function succeeds. Any errors or exceptions are reraised, and errors in the transaction SQL commands raise an exception as well.
Examples
iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.transaction db do
...> Sqlcx.exec(db, "create table foo(id integer)")
...> Sqlcx.exec(db, "insert into foo (id) values(42)")
...> end
iex> Sqlcx.query_rows(db, "select * from foo")
{:ok, [%{id: 42}]}
Specs
with_db(String.t(), (connection() -> t), open_options()) :: {:ok, t} | error() when t: var
Opens a database connection, executes fun
on it and closes it.
Returns an {:ok, result}
tuple containing the return value of fun
or an
error/0
tuple if an error occurs while opening the database connection.
Note that errors raised by fun
will remain raised and will not be converted
to an error tuple.
Specs
with_db!(String.t(), (connection() -> t), open_options()) :: t when t: var
Opens a database connection, executes fun
on it and closes it.
Returns the result of fun
or raises an error if the opening the database fails.
Specs
with_savepoint(connection(), (... -> t), db_timeout: timeout(), name: String.t(), args: [any()] ) :: t when t: var
Runs fun
inside a SAVEPOINT
transaction. Unlike regular BEGIN
/COMMIT
transactions, savepoints can be nested.
fun
will be called with the database connection as its first argument. Use
the args
option to pass a list of further arguments if needed.
If fun
returns without raising an exception, the savepoint will be commited
via RELEASE SAVEPOINT
. Otherwise, ROLLBACK TO SAVEPOINT
will be called.
Even if an inner savepoint is committed, it may be rolled back if an outer
transaction is rolled back.
By default, a random savepoint name is automatically generated. Pass a name
option to set your own savepoint name. Be careful with the naming of savepoints,
unexpected things might happen if you nest savepoints of the same name. The
name
is also not escaped in the SQL commands, so it should only use
alphanumeric characters.
See also the sqlite3 docs about savepoints
Returns the result of fun
if the function succeeds. If any exceptions are
raised or values are thrown, the savepoint transaction will be rolled back and
the exception will be reraised or the value rethrown. This implies that if you
nest savepoint transactions, unless you rescue/catch the exception, the outer
transaction will also be rolled back because an exception has been raised
Examples
iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.with_savepoint(db, fn(db, value) ->
...> Sqlcx.exec(db, "create table foo (id integer)")
...> {:ok, _} = Sqlcx.query(db, "insert into foo (id) values (?)", bind: [value])
...> try do
...> Sqlcx.with_savepoint(db, fn(db) ->
...> Sqlcx.exec(db, "insert into foo (id) values (3)")
...> :ok = Sqlcx.exec(db, "not a valid sql query") # Raises an exception
...> end) # will be rolled back because the function fails
...> rescue
...> _ -> "we don't care about the exception"
...> end
...> end, [args: [42]])
iex> Sqlcx.query_rows(db, "select * from foo")
{:ok, [%{id: 42}]}
Specs
with_transaction(connection(), (... -> t), db_timeout: timeout(), args: [any()]) :: t when t: var
Runs fun
inside a transaction. If fun
returns without raising an exception,
the transaction will be commited via COMMIT
. Otherwise, ROLLBACK
will be called.
fun
will be called with the database connection as its first argument. Use
the args
option to pass a list of further arguments if needed.
Note that transactions cannot be nested. Prefer with_savepoint/3
which also
allows nested transactions.
See also the transaction/2
version which takes a code block.
Returns the result of the function if it function succeeds. Any errors or exceptions are reraised, and errors in the transaction SQL commands raise an exception as well.
Example
iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.with_transaction(db, fn(db, val) ->
...> Sqlcx.exec(db, "create table foo(id integer)")
...> {:ok, _} = Sqlcx.query(db, "insert into foo (id) values(:value)", bind: [value: val])
...> end, [args: [42]])
iex> Sqlcx.query_rows(db, "select * from foo")
{:ok, [%{id: 42}]}