Sqlcx (sqlcx v2.0.0-rc.1) View Source

Sqlcx gives you a way to create and query SQLCipher (encrypted SQLite) databases.

Basic Example

iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.exec(db, "CREATE TABLE t (a INTEGER, b INTEGER, c INTEGER)")
:ok
iex> Sqlcx.exec(db, "INSERT INTO t VALUES (1, 2, 3), (4, 5, 6)")
:ok
iex> Sqlcx.query(db, "SELECT * FROM t")
{:ok, [[1, 2, 3], [4, 5, 6]]}
iex> Sqlcx.query_rows(db, "SELECT * FROM t")
{:ok, [%{a: 1, b: 2, c: 3}, %{a: 4, b: 5, c: 6}]}

Configuration

Sqlcx uses the Erlang library esqlcipher which accepts a timeout parameter for almost all interactions with the database. The default value for this timeout is 5000 ms. Many functions in Sqlcx accept a :db_timeout option that is passed on to the esqlcipher calls and also defaults to 5000 ms. If required, this default value can be overridden globally with the following in your config.exs:

config :sqlcx, db_timeout: 10_000 # or other positive integer number of ms

Another esqlcipher parameter is :db_chunk_size. This is the number of rows that are read from native sqlcipher and send to the Erlang process in one bulk. For example, consider a table mytable that has 1000 rows. We make the query to get all rows with db_chunk_size: 500 parameter:

Sqlcx.query(db, "select * from mytable", db_chunk_size: 500)

In this case, all rows will be passed from native sqlcipher OS thread to the Erlang process in two passes with each pass containing 500 rows. This reduces the overhead of transmitting and copying rows from the native OS sqlcipher thread to the Erlang process by chunking the list of result rows. There will be no difference visible to the end user: The result of the Sqlcx.query call will be the same no matter the chunk size. Please decrease :db_chunk_size if rows are heavy. The default value is 5000. If you’re in doubt what to do with this parameter, just leave it as is. The default value will be ok in almost all cases. You can also change the default value like this:

config :sqlcx, db_chunk_size: 500 # if most of the database rows are heavy

Link to this section Summary

Types

Database connection to an encrypted or unencrypted database.

Database connection to an encrypted database., subtype of connection()

An error that can be returned by sqlcipher calls. Contains an error type atom and a reason string.

Options that can be passed to open/2 or open!/2 for encrypted databases.

Options that can be passed to open/2, open!/2, or with_db/3.

Options that can be passed to open/2 or open!/2 for unencrypted databases.

Database password

Database connection to an unencrypted database, subtype of connection()

Options that can be passed to the query functions.

Functions

Return the number of the rows that have been modified, inserted, or deleted by the last statement.

Close the database connection.

Send a raw SQL statement to the database.

Same as exec/3 but raises a Sqlcx.QueryError on error. Returns :ok otherwise.

Whether conn is a database connection. Can be used in guards.

Whether conn is a connection to an encrypted database. Can be used in guards.

Whether conn is a connection to an unencrypted/plaintext database. Can be used in guards.

Open an sqlite3/sqlcipher database connection that can be queried. If the database file does not yet exist, it will be created.

Shortcut for open/2, raises errors instead of returning error tuples.

Runs a query and returns the results.

Same as query/3 but raises a Sqlcx.QueryError on error.

Runs a query and returns the results as a list of rows each represented as a map.

Same as query_rows/3 but raises a Sqlcx.QueryError on error.

Change the password of an encrypted database.

Shortcut for rekey/3, raises errors instead of returning error tuples.

Run a code block inside a SAVEPOINT transaction. Unlike regular BEGIN/COMMIT transactions, savepoints can be nested.

Sets a PID to receive notifications about table updates.

Run a code block inside a transaction. If the do block does not raise an exception, the transaction will be commited via COMMIT. Otherwise, ROLLBACK will be called.

Opens a database connection, executes fun on it and closes it.

Opens a database connection, executes fun on it and closes it.

Runs fun inside a SAVEPOINT transaction. Unlike regular BEGIN/COMMIT transactions, savepoints can be nested.

Runs fun inside a transaction. If fun returns without raising an exception, the transaction will be commited via COMMIT. Otherwise, ROLLBACK will be called.

Link to this section Types

Specs

connection()

Database connection to an encrypted or unencrypted database.

Link to this opaque

encrypted_connection()

View Source (opaque)

Specs

encrypted_connection()

Database connection to an encrypted database., subtype of connection()

Specs

error() :: {:error, {id :: atom(), reason :: charlist()}}

An error that can be returned by sqlcipher calls. Contains an error type atom and a reason string.

Link to this type

open_encrypted_options()

View Source

Specs

open_encrypted_options() ::
  [db_password: password(), db_timeout: timeout()]
  | [db_timeout: timeout(), db_password: password()]
  | [{:db_password, password()}]

Options that can be passed to open/2 or open!/2 for encrypted databases.

Specs

open_options() :: [db_password: password(), db_timeout: timeout()]

Options that can be passed to open/2, open!/2, or with_db/3.

Link to this type

open_plaintext_options()

View Source

Specs

open_plaintext_options() :: [{:db_timeout, timeout()}]

Options that can be passed to open/2 or open!/2 for unencrypted databases.

Specs

password() :: String.t() | {:raw, binary()}

Database password

Link to this opaque

plaintext_connection()

View Source (opaque)

Specs

plaintext_connection()

Database connection to an unencrypted database, subtype of connection()

Specs

query_options() :: [
  bind: [Sqlcx.Statement.bind_value()],
  into: Collectable.t(),
  db_timeout: timeout(),
  db_chunk_size: pos_integer()
]

Options that can be passed to the query functions.

Link to this section Functions

Specs

changes(connection(), [{:db_timeout, timeout()}]) :: non_neg_integer()

Return the number of the rows that have been modified, inserted, or deleted by the last statement.

See https://www.sqlite.org/c3ref/changes.html for details information.

Example

iex> {:ok, db} = Sqlcx.open(":memory:") iex> Sqlcx.exec(db, "create table test (val integer)") :ok iex> Sqlcx.exec(db, "insert into test(val) values (1), (2)") :ok iex> Sqlcx.changes(db) 2

Specs

close(connection(), [{:db_timeout, timeout()}]) :: :ok | error()

Close the database connection.

Link to this function

exec(db, sql, opts \\ [])

View Source

Specs

exec(connection(), String.t(), [{:db_timeout, timeout()}]) :: :ok | error()

Send a raw SQL statement to the database.

This function is intended for running simple/one-off SQL statements.

Parameters are not supported here, see query/3 related functions for those! No results are returned; queries return either :ok or an error.

This is generally useful for things like re-playing a SQL export back into the database or for simple commands like BEGIN or COMMIT (but see also with_transaction/3 and with_savepoint/4 for transactions).

Do not manually insert values into SQL statements as that is prone to SQL injection. If you need the results returned by a query, use the Sqlcx.Query interface (see also the query/3 and query_rows/3 shortcuts) or work directly with prepared statements using Sqlcx.Statement.

Link to this function

exec!(db, sql, opts \\ [])

View Source

Specs

exec!(connection(), String.t(), [{:db_timeout, timeout()}]) :: :ok

Same as exec/3 but raises a Sqlcx.QueryError on error. Returns :ok otherwise.

Whether conn is a database connection. Can be used in guards.

Link to this macro

is_encrypted(conn)

View Source (macro)

Whether conn is a connection to an encrypted database. Can be used in guards.

Link to this macro

is_plaintext(conn)

View Source (macro)

Whether conn is a connection to an unencrypted/plaintext database. Can be used in guards.

Specs

Open an sqlite3/sqlcipher database connection that can be queried. If the database file does not yet exist, it will be created.

Pass [db_password: "1234"] as an options to open an sqlcipher encrypted database. If the database does not exist, an encrypted database will be created on first write. The actual encryption key will be derived from db_password using PBKDF2 key derivation by sqlcipher. However, it's possible to specify a raw byte sequence as a key using [db_password: {:raw, <<...>>}]. The raw key has to be either 32 or 48 bytes long where the key is 32 bytes and the optional salt is another 16 bytes long. If the salt is not explicitly provided, it will be generated randomly and stored in the first 16 bytes of the database. The raw binary key will be internally hex-encoded and passed to sqlcipher as x'A0B1C2(...)D3E4F5' with either 64 or 96 hex chaarcters. In order to avoid confusion or unsafe behavior, string passwords of that format will return an {:error, {:badarg, _}}.

Please refer to the sqlcipher documentation for further information about the generation and usage of encryption keys. Note that opening encrypted databases can take quite some time. This is by design because key derivation uses the PBKDF2 algorithm. If on a slower machine, adjust the timeout accordingly.

Returns {:ok, db} where db is a database connection if the file could be opened or created and could be read (i.e., the db_password was correct or the database is not encrypted).

If the password is incorrect or a password was given on an unencrypted database (or vice versa), or if the file is corrupted or unreadable, will return an error of the form {:error, {:baddb, _}}.

Remember to close/2 the database connection when it is no longer needed (although it will automatically be closed when its process dies or it is garbage collected)

Specs

Shortcut for open/2, raises errors instead of returning error tuples.

Link to this function

query(db, sql, opts \\ [])

View Source

Specs

query(connection(), String.t(), query_options()) :: {:ok, [keyword()]} | error()

Runs a query and returns the results.

Parameters

  • db - An SQLCipher database connection.
  • sql - The query to run as a string.
  • opts - Options to pass into the query. See below for details.

Options

  • bind - List/keyword list of values to bind to the sqlite parameters. See Statement.bind/3 for details.
  • into - The collection to put each row into. If provided, values are ziped with the column names, so into: %{} produces a Map (note that column names are Strings, not atoms). Defaults to nil which simply outputs each row as a list of values without any column names.
  • db_timeout - The timeout (in ms) to apply to each of the underlying SQLCipher operations. Defaults to Application.get_env(:sqlcx, :db_timeout) or 5000 ms if not configured.
  • db_chunk_size - The number of rows to read from native sqlite and send to erlang process in one bulk. Defaults to Application.get_env(:sqlcx, :db_chunk_size) or 5000 if not configured.

Returns

  • {:ok, [[values...]...]} on success
  • {:error, _} on failure.

Example

iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.exec(db, "CREATE TABLE data (cat, name)")
:ok
iex> Sqlcx.exec(db, "INSERT INTO data VALUES (1, 'a'), (2, 'b'), (2, 'c')")
:ok
iex> Sqlcx.query(db, "SELECT * FROM data WHERE cat = :cat", bind: [cat: 2])
{:ok, [[2, "b"], [2, "c"]]}
iex> Sqlcx.query(db, "SELECT * FROM data WHERE cat = :cat", bind: [cat: 2], into: [])
{:ok, [[{"cat", 2}, {"name", "b"}], [{"cat", 2}, {"name", "c"}]]}
iex> Sqlcx.query(db, "SELECT * FROM data WHERE cat = :cat", bind: [cat: 2], into: %{})
{:ok, [%{"cat" => 2, "name" => "b"}, %{"cat" => 2, "name" => "c"}]}
iex> Sqlcx.close(db)
:ok
Link to this function

query!(db, sql, opts \\ [])

View Source

Specs

query!(connection(), String.t(), query_options()) :: [Enum.t()]

Same as query/3 but raises a Sqlcx.QueryError on error.

Returns the results otherwise.

Link to this function

query_rows(db, sql, opts \\ [])

View Source

Specs

query_rows(connection(), String.t(), query_options()) :: {:ok, %{}} | error()

Runs a query and returns the results as a list of rows each represented as a map.

Unlike query/3, column names are translated into atoms and rows are Maps so values can be accessed idiomatically using a row.value syntax.

Questioning: This function might be dropped in future versions.

Parameters

  • db - An SQLCipher database.
  • sql - The query to run as a string.
  • opts - Options to pass into the query. See below for details.

Options

  • bind - List/keyword list of values to bind to the sqlite parameters. See Statement.bind/3 for details.
  • into - The collection to put each row into. Use [] to get keyword lists. Defaults to %{}.
  • db_timeout - The timeout (in ms) to apply to each of the underlying SQLCipher operations. Defaults to Application.get_env(:sqlcx, :db_timeout) or 5000 ms if not configured.
  • db_chunk_size - The number of rows to read from native sqlite and send to erlang process in one bulk. Defaults to Application.get_env(:sqlcx, :db_chunk_size) or 5000 if not configured.

Returns

  • {:ok, [%{name: value}]} on success
  • {:error, _} on failure.

Example

iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.exec(db, "CREATE TABLE data (cat, name)")
:ok
iex> Sqlcx.exec(db, "INSERT INTO data VALUES (1, 'a'), (2, 'b'), (2, 'c')")
:ok
iex> Sqlcx.query_rows(db, "SELECT * FROM data WHERE cat = :cat", bind: [cat: 2])
{:ok, [%{cat: 2, name: "b"}, %{cat: 2, name: "c"}]}
iex> Sqlcx.query_rows(db, "SELECT * FROM data WHERE cat = :cat", bind: [cat: 2], into: [])
{:ok, [[cat: 2, name: "b"], [cat: 2, name: "c"]]}
iex> Sqlcx.close(db)
:ok
Link to this function

query_rows!(db, sql, opts \\ [])

View Source

Specs

query_rows!(connection(), String.t(), query_options()) :: %{}

Same as query_rows/3 but raises a Sqlcx.QueryError on error.

Returns the results otherwise.

Link to this function

rekey(db, password, opts \\ [])

View Source

Specs

rekey(encrypted_connection(), String.t(), [{:db_timeout, timeout()}]) ::
  :ok | error()

Change the password of an encrypted database.

password can either be a String or a {:raw, <<...>>} tuple containing a binary raw key. See open/2.

Note that this function cannot be used to encrypt an unencrypted database.

Returns :ok or an error/0 if the new key is invalid or rekeying the database fails.

Link to this function

rekey!(db, password, opts \\ [])

View Source

Specs

rekey!(encrypted_connection(), String.t(), [{:db_timeout, timeout()}]) :: :ok

Shortcut for rekey/3, raises errors instead of returning error tuples.

Link to this macro

savepoint(db, list)

View Source (macro)

Run a code block inside a SAVEPOINT transaction. Unlike regular BEGIN/COMMIT transactions, savepoints can be nested.

If the do block returns without raising an exception, the savepoint will be commited via RELEASE SAVEPOINT. Otherwise, ROLLBACK TO SAVEPOINT will be called. Even if an inner savepoint is committed, it may be rolled back if an outer transaction is rolled back.

Since this is implemented as a macro, Sqlcx must be required or imported in order to use it. See also the with_savepoint/3 version which takes a function as an argument.

Note that this version does not take any further arguments. Thus, the savepoint will be automatically given a random name and the default timeout will be applied to the SAVEPOINT, RELEASE and ROLLBACK commands.

Returns {:ok, result} if the function succeeds, {:exception, err} if the function raises an exception, {:error, err} if any of the savepoint sqlite commands fails.

Examples

iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.savepoint db do
...>   Sqlcx.exec(db, "create table foo(id integer)")
...>   Sqlcx.exec(db, "insert into foo (id) values(42)")
...>   try do
...>     Sqlcx.savepoint db do
...>       Sqlcx.exec(db, "insert into foo (id) values (3)")
...>       :ok = Sqlcx.exec(db, "not a valid sql query") # Raises an exception
...>     end # will be rolled back because the do block fails
...>   rescue
...>     _ -> "we don't care about the exception"
...>   end
...> end
iex> Sqlcx.query_rows(db, "select * from foo")
{:ok, [%{id: 42}]}
Link to this function

set_update_hook(db, pid, opts \\ [])

View Source

Specs

set_update_hook(connection(), pid(), [{:db_timeout, timeout()}]) ::
  :ok | error()

Sets a PID to receive notifications about table updates.

Messages will come in the shape of: {action, table, rowid}

  • action: either :insert, :update, or :delete
  • table: charlist of the table name. Example: 'posts'
  • rowid: internal rowid index of the row. If the table contains an INTEGER PRIMARY KEY column, this column will be the row id. Do note that INT PRIMARY KEY and similar will not be an alias for the row id.

Also note that update hooks do not work on WITHOUT ROWID tables.

See the sqlite3 docs for update_hook and for row ids.

Link to this macro

transaction(db, list)

View Source (macro)

Run a code block inside a transaction. If the do block does not raise an exception, the transaction will be commited via COMMIT. Otherwise, ROLLBACK will be called.

Note that transactions cannot be nested. If you need nested transactions, use savepoint/2.

Since this is implemented as a macro, Sqlcx must be required or imported in order to use it. See also the with_transaction/3 version which takes a function as an argument.

Note that this version does not take any further arguments. The default timeout will be applied to the BEGIN, COMMIT and ROLLBACK commands.

Returns the result of the function if it function succeeds. Any errors or exceptions are reraised, and errors in the transaction SQL commands raise an exception as well.

Examples

iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.transaction db do
...>   Sqlcx.exec(db, "create table foo(id integer)")
...>   Sqlcx.exec(db, "insert into foo (id) values(42)")
...> end
iex> Sqlcx.query_rows(db, "select * from foo")
{:ok, [%{id: 42}]}
Link to this function

with_db(path, fun, opts \\ [])

View Source

Specs

with_db(String.t(), (connection() -> t), open_options()) :: {:ok, t} | error()
when t: var

Opens a database connection, executes fun on it and closes it.

Returns an {:ok, result} tuple containing the return value of fun or an error/0 tuple if an error occurs while opening the database connection. Note that errors raised by fun will remain raised and will not be converted to an error tuple.

Link to this function

with_db!(path, fun, opts \\ [])

View Source

Specs

with_db!(String.t(), (connection() -> t), open_options()) :: t when t: var

Opens a database connection, executes fun on it and closes it.

Returns the result of fun or raises an error if the opening the database fails.

Link to this function

with_savepoint(db, fun, opts \\ [])

View Source

Specs

with_savepoint(connection(), (... -> t),
  db_timeout: timeout(),
  name: String.t(),
  args: [any()]
) :: t
when t: var

Runs fun inside a SAVEPOINT transaction. Unlike regular BEGIN/COMMIT transactions, savepoints can be nested.

fun will be called with the database connection as its first argument. Use the args option to pass a list of further arguments if needed.

If fun returns without raising an exception, the savepoint will be commited via RELEASE SAVEPOINT. Otherwise, ROLLBACK TO SAVEPOINT will be called. Even if an inner savepoint is committed, it may be rolled back if an outer transaction is rolled back.

By default, a random savepoint name is automatically generated. Pass a name option to set your own savepoint name. Be careful with the naming of savepoints, unexpected things might happen if you nest savepoints of the same name. The name is also not escaped in the SQL commands, so it should only use alphanumeric characters.

See also the sqlite3 docs about savepoints

Returns the result of fun if the function succeeds. If any exceptions are raised or values are thrown, the savepoint transaction will be rolled back and the exception will be reraised or the value rethrown. This implies that if you nest savepoint transactions, unless you rescue/catch the exception, the outer transaction will also be rolled back because an exception has been raised

Examples

iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.with_savepoint(db, fn(db, value) ->
...>   Sqlcx.exec(db, "create table foo (id integer)")
...>   {:ok, _} = Sqlcx.query(db, "insert into foo (id) values (?)", bind: [value])
...>   try do
...>     Sqlcx.with_savepoint(db, fn(db) ->
...>       Sqlcx.exec(db, "insert into foo (id) values (3)")
...>       :ok = Sqlcx.exec(db, "not a valid sql query") # Raises an exception
...>     end) # will be rolled back because the function fails
...>   rescue
...>     _ -> "we don't care about the exception"
...>   end
...> end, [args: [42]])
iex> Sqlcx.query_rows(db, "select * from foo")
{:ok, [%{id: 42}]}
Link to this function

with_transaction(db, fun, opts \\ [])

View Source

Specs

with_transaction(connection(), (... -> t), db_timeout: timeout(), args: [any()]) ::
  t
when t: var

Runs fun inside a transaction. If fun returns without raising an exception, the transaction will be commited via COMMIT. Otherwise, ROLLBACK will be called.

fun will be called with the database connection as its first argument. Use the args option to pass a list of further arguments if needed.

Note that transactions cannot be nested. Prefer with_savepoint/3 which also allows nested transactions.

See also the transaction/2 version which takes a code block.

Returns the result of the function if it function succeeds. Any errors or exceptions are reraised, and errors in the transaction SQL commands raise an exception as well.

Example

iex> {:ok, db} = Sqlcx.open(":memory:")
iex> Sqlcx.with_transaction(db, fn(db, val) ->
...>   Sqlcx.exec(db, "create table foo(id integer)")
...>   {:ok, _} = Sqlcx.query(db, "insert into foo (id) values(:value)", bind: [value: val])
...> end, [args: [42]])
iex> Sqlcx.query_rows(db, "select * from foo")
{:ok, [%{id: 42}]}