Xandra v0.7.2 Xandra
This module provides the main API to interface with Cassandra.
This module handles the connection to Cassandra, queries, connection pooling,
connection backoff, logging, and more. Many of these features are provided by
the DBConnection
library, which
Xandra is built on top of.
Errors
Many of the functions in this module (whose names don’t end with a !
)
return values in the form {:ok, result}
or {:error, error}
. While result
varies based on the specific function, error
is always one of the following:
a
Xandra.Error
struct: such structs represent errors returned by Cassandra. When such an error is returned, it means that communicating with the Cassandra server was successful, but the server returned an error. Examples of these errors are syntax errors in queries, non-existent tables, and so on. SeeXandra.Error
for more information.a
Xandra.ConnectionError
struct: such structs represent errors in the communication with the Cassandra server. For example, if the Cassandra server dies while the connection is waiting for a response from the server, aXandra.ConnectionError
error will be returned. SeeXandra.ConnectionError
for more information.
Parameters, encoding, and types
Xandra supports parameterized queries (queries that specify “parameter” values
through ?
or :named_value
):
SELECT * FROM users WHERE name = ? AND email = ?
SELECT * FROM users WHERE name = :name AND email = :email
When a query has positional parameters, parameters can be passed as a list to
functions like execute/4
: in this case, a parameter in a given position in
the list will be used as the ?
in the corresponding position in the
query. When a query has named parameters, parameters are passed as a map with
string keys representing each parameter’s name and values representing the
corresponding parameter’s value.
Types
Cassandra supports many types of values, and some types have “shades” that cannot be represented by Elixir types. For example, in Cassandra an integer could be a “bigint” (a 64 bit integer), an “int” (a 32 bit integer), a “smallint” (a 16 bit integer), or others; in Elixir, however, integers are just integers (with varying size to be precise), so it is impossible to univocally map Elixir integers to a specific Cassandra integer type. For this reason, when executing simple parameterized queries (statements) it is necessary to explicitly specify the type of each value.
To specify the type of a value, that value needs to be provided as a
two-element tuple where the first element is the value’s type and the second
element is the value itself. Types are expressed with the same syntax used in
CQL: for example, 16-bit integers are represented as "smallint"
, while maps
of strings to booleans are represented as "map<text, boolean>"
.
# Using a list of parameters:
statement = "INSERT INTO species (name, properties) VALUES (?, ?)"
Xandra.execute(conn, statement, [
{"text", "human"},
{"map<text, boolean>", %{"legs" => true, "arms" => true, "tail" => false}},
])
# Using a map of parameters:
statement = "INSERT INTO species (name, properties) VALUES (:name, :properties)"
Xandra.execute(conn, statement, %{
"name" => {"text", "human"},
"properties" => {"map<text, boolean>", %{"legs" => true, "arms" => true, "tail" => false}},
})
You only need to specify types for simple queries (statements): when using prepared queries, the type information of each parameter of the query is encoded in the prepared query itself.
# Using a map of parameters:
prepared = Xandra.prepare!(conn, "INSERT INTO species (name, properties) VALUES (:name, :properties)")
Xandra.execute(conn, prepared, %{
"name" => "human",
"properties" => %{"legs" => true, "arms" => true, "tail" => false},
})
User-defined types
Xandra supports user-defined types (UDTs). A UDT can be inserted as a map with string fields. For example, consider having created the following UDTs:
CREATE TYPE full_name (first_name text, last_name text)
CREATE TYPE profile (username text, full_name frozen<full_name>)
and having the following table:
CREATE TABLE users (id int PRIMARY KEY, profile frozen<profile>)
Inserting rows will look something like this:
prepared_insert = Xandra.prepare!(conn, "INSERT INTO users (id, profile) VALUES (?, ?)")
profile = %{
"username" => "bperry",
"full_name" => %{"first_name" => "Britta", "last_name" => "Perry"},
}
Xandra.execute!(conn, prepared_insert, [_id = 1, profile])
Note that inserting UDTs is only supported on prepared queries.
When retrieved, UDTs are once again represented as maps with string keys. Retrieving the row inserted above would look like this:
%{"profile" => profile} = conn |> Xandra.execute!("SELECT id, profile FROM users") |> Enum.fetch!(0)
profile
#=> %{"username" => "bperry", "full_name" => %{"first_name" => "Britta", "last_name" => "Perry"}}
Reconnections
Thanks to the DBConnection
library, Xandra is able to handle connection
losses and to automatically reconnect to Cassandra. By default, reconnections
are retried at exponentially increasing randomized intervals, but backoff can
be configured through a subset of the options accepted by
start_link/2
. These options are described in the documentation for
DBConnection.start_link/2
.
Clustering
Xandra supports connecting to multiple nodes in a Cassandra cluster and
executing queries on different nodes based on load balancing strategies. See
the documentation for Xandra.Cluster
for more information.
Authentication
Xandra supports Cassandra authentication. See the documentation for
Xandra.Authenticator
for more information.
Retrying failed queries
Xandra takes a customizable and extensible approach to retrying failed queries
through “retry strategies” that encapsulate the logic for retrying
queries. See Xandra.RetryStrategy
for documentation on retry strategies.
Compression
Xandra supports compression. To inform the Cassandra server that the
connections you start should use compression for data transmitted to and from
the server, you can pass the :compressor
option to start_link/1
; this
option should be a module that implements the Xandra.Compressor
behaviour. After this, all compressed data that Cassandra sends to the
connection will be decompressed using this behaviour module.
To compress outgoing data (such as when issuing or preparing queries), the
:compressor
option should be specified explicitly. When it’s specified, the
given module will be used to compress data. If no :compressor
option is
passed, the outgoing data will not be compressed.
Summary
Functions
Executes the given simple query, prepared query, or batch query
Executes the given simple query or prepared query with the given parameters
Executes the given simple query, prepared query, or batch query, raising if there’s an error
Executes the given simple query, prepared query, or batch query, raising if there’s an error
Prepares the given query
Prepares the given query, raising if there’s an error
Acquires a locked connection from conn
and executes fun
passing such
connection as the argument
Starts a new connection or pool of connections to Cassandra
Streams the results of a simple query or a prepared query with the given params
Types
Functions
execute(conn, statement | Xandra.Prepared.t, values) :: {:ok, result} | {:error, error}
execute(conn, Xandra.Batch.t, Keyword.t) :: {:ok, Xandra.Void.t} | {:error, error}
Executes the given simple query, prepared query, or batch query.
Returns {:ok, result}
if executing the query was successful, or {:error,
error}
otherwise. The meaning of the params_or_options
argument depends on
what query
is:
if
query
is a batch query, thanparams_or_options
has to be a list of options that will be used to run the batch query (since batch queries don’t use parameters as parameters are attached to each query in the batch).if
query
is a simple query (a string) or a prepared query, thenparams_or_opts
is a list or map of parameters, and this function is exactly the same as callingexecute(conn, query, params_or_options, [])
.
When query
is a batch query, successful results will always be Xandra.Void
structs.
When {:error, error}
is returned, error
can be either a Xandra.Error
or
a Xandra.ConnectionError
struct. See the module documentation for more
information on errors.
Options for batch queries
When query
is a batch query, params_or_options
is a list of options. All
options supported by DBConnection.execute/4
are supported, and the following
additional batch-specific options:
:consistency
- same as the:consistency
option described in the documentation forexecute/4
.:serial_consistency
- same as the:serial_consistency
option described in the documentation forexecute/4
.:timestamp
- (integer) using this option means that the provided timestamp will apply to all the statements in the batch that do not explicitly specify a timestamp.
Examples
For examples on executing simple queries or prepared queries, see the
documentation for execute/4
. Examples below specifically refer to batch
queries. See the documentation for Xandra.Batch
for more information about
batch queries and how to construct them.
prepared_insert = Xandra.prepare!(conn, "INSERT (email, name) INTO users VALUES (?, ?)")
batch =
Xandra.Batch.new()
|> Xandra.Batch.add(prepared_insert, ["abed@community.com", "Abed Nadir"])
|> Xandra.Batch.add(prepared_insert, ["troy@community.com", "Troy Barnes"])
|> Xandra.Batch.add(prepared_insert, ["britta@community.com", "Britta Perry"])
# Execute the batch:
Xandra.execute(conn, batch)
#=> {:ok, %Xandra.Void{}}
# Execute the batch with a default timestamp for all statements:
Xandra.execute(conn, batch, timestamp: System.system_time(:millisecond) - 1_000)
#=> {:ok, %Xandra.Void{}}
All DBConnection.execute/4
options are supported here as well:
Xandra.execute(conn, batch, pool: DBConnection.Poolboy)
#=> {:ok, %Xandra.Void{}}
Executes the given simple query or prepared query with the given parameters.
Returns {:ok, result}
where result
is the result of executing query
if
the execution is successful (there are no network errors or semantic errors
with the query), or {:error, error}
otherwise.
result
can be one of the following:
a
Xandra.Void
struct - returned for queries such asINSERT
,UPDATE
, orDELETE
.a
Xandra.SchemaChange
struct - returned for queries that perform changes on the schema (such as creating tables).a
Xandra.SetKeyspace
struct - returned forUSE
queries.a
Xandra.Page
struct - returned for queries that return rows (such asSELECT
queries).
The properties of each of the results listed above are described in each result’s module.
Options
This function accepts all options accepted by DBConnection.execute/4
, plus
the following ones:
:consistency
- (atom) specifies the consistency level for the given query. See the Cassandra documentation for more information on consistency levels. The value of this option can be one of::one
(default):two
:three
:any
:quorum
:all
:local_quorum
:each_quorum
:serial
:local_serial
:local_one
:page_size
- (integer) The size of a page of results. Ifquery
returnsXandra.Page
struct, that struct will contain at most:page_size
rows in it. Defaults to10_000
.:cursor
- (Xandra.Page
struct) the offset where rows should be returned from. By default this option is not present and paging starts from the beginning. See the “Paging” section below for more information on how to page queries.:timestamp
- (integer) the default timestamp for the query (in microseconds). If provided, overrides the server-side assigned timestamp; however, a timestamp in the query itself will still override this timestamp.:serial_consistency
- (atom) specifies the serial consistency to use for executing the given query. Can be of:serial
and:local_serial
.:compressor
- (module) the compressor module used to compress and decompress data. See the “Compression” section in the module documentation. By default, this option is not present.:retry_strategy
- (module) the module implementing theXandra.RetryStrategy
behaviour that is used in case the query fails to determine whether to retry it or not. See the “Retrying failed queries” section in the module documentation. By default, this option is not present.
Parameters
The params
argument specifies parameters to use when executing the query; it
can be either a list of positional parameters (specified via ?
in the query)
or a map of named parameters (specified as :named_parameter
in the
query). When query
is a simple query, the value of each parameter must be a
two-element tuple specifying the type used to encode the value and the value
itself; when query
is a prepared query, this is not necessary (and values
can just be values) as the type information is encoded in the prepared
query. See the module documenatation for more information about query
parameters, types, and encoding values.
Examples
Executing a simple query (which is just a string):
statement = "INSERT INTO users (first_name, last_name) VALUES (:first_name, :last_name)"
{:ok, %Xandra.Void{}} = Xandra.execute(conn, statement, %{
"first_name" => {"text", "Chandler"},
"last_name" => {"text", "Bing"},
})
Executing a prepared query:
prepared = Xandra.prepare!(conn, "INSERT INTO users (first_name, last_name) VALUES (?, ?)")
{:ok, %Xandra.Void{}} = Xandra.execute(conn, prepared, ["Monica", "Geller"])
Performing a SELECT
query and using Enum.to_list/1
to convert the
Xandra.Page
result to a list of rows:
{:ok, %Xandra.Page{} = page} = Xandra.execute(conn, "SELECT * FROM users", _params = [])
Enum.to_list(page)
#=> [%{"first_name" => "Chandler", "last_name" => "Bing"},
#=> %{"first_name" => "Monica", "last_name" => "Geller"}]
Ensuring the write is written to the commit log and memtable of at least three replica nodes:
statement = "INSERT INTO users (first_name, last_name) VALUES ('Chandler', 'Bing')"
{:ok, %Xandra.Void{}} = Xandra.execute(conn, statement, _params = [], consistency: :three)
This function supports all options supported by DBConnection.execute/4
; for
example, if the conn
connection was started with pool: DBConnection.Poolboy
,
then the :pool
option would have to be passed here as well:
statement = "DELETE FROM users WHERE first_name = 'Chandler'"
{:ok, %Xandra.Void{}} = Xandra.execute(conn, statement, _params = [], pool: DBConnection.Poolboy)
Paging
Since execute/4
supports the :cursor
option, it is possible to manually
implement paging. For example, given the following prepared query:
prepared = Xandra.prepare!(conn, "SELECT first_name FROM users")
We can now execute such query with a specific page size using the :page_size
option:
{:ok, %Xandra.Page{} = page} = Xandra.execute(conn, prepared, [], page_size: 2)
Since :page_size
is 2
, page
will contain at most 2
rows:
Enum.to_list(page)
#=> [%{"first_name" => "Ross"}, %{"first_name" => "Rachel"}]
Now, we can pass page
as the value of the :cursor
option to let the paging
start from where we left off:
{:ok, %Xandra.Page{} = new_page} = Xandra.execute(conn, prepared, [], page_size: 2, cursor: page)
Enum.to_list(page)
#=> [%{"first_name" => "Joey"}, %{"first_name" => "Phoebe"}]
However, using :cursor
and :page_size
directly with execute/4
is not
recommended when the intent is to “stream” a query. For that, it’s recommended
to use stream_pages!/4
. Also note that if the Xandra.Page
struct provided
as :cursor
shows there are no more pages to fetch, an ArgumentError
exception will be raised; be sure to check for this with
Xandra.Page.more_pages_available?/1
.
execute!(conn, statement | Xandra.Prepared.t, values) :: result | no_return
execute!(conn, Xandra.Batch.t, Keyword.t) :: Xandra.Void.t | no_return
Executes the given simple query, prepared query, or batch query, raising if there’s an error.
This function behaves exactly like execute/3
, except that it returns
successful results directly and raises on errors.
Examples
Xandra.execute!(conn, "INSERT INTO users (name, age) VALUES ('Jane', 29)")
#=> %Xandra.Void{}
Executes the given simple query, prepared query, or batch query, raising if there’s an error.
This function behaves exactly like execute/4
, except that it returns
successful results directly and raises on errors.
Examples
statement = "INSERT INTO users (name, age) VALUES ('John', 43)"
Xandra.execute!(conn, statement, _params = [], consistency: :quorum)
#=> %Xandra.Void{}
prepare(conn, statement, Keyword.t) :: {:ok, Xandra.Prepared.t} | {:error, error}
Prepares the given query.
This function prepares the given statement on the Cassandra server. If
preparation is successful and there are no network errors while talking to the
server, {:ok, prepared}
is returned, otherwise {:error, error}
is
returned.
The returned prepared query can be run through execute/4
, or used inside a
batch (see Xandra.Batch
).
Errors returned by this function can be either Xandra.Error
or
Xandra.ConnectionError
structs. See the module documentation for more
information about errors.
Supports all the options supported by DBConnection.prepare/3
, and the
following additional options:
:force
- (boolean) whentrue
, forces the preparation of the query on the server instead of trying to read the prepared query from cache. See the “Prepared queries cache” section below. Defaults tofalse
.:compressor
- (module) the compressor module used to compress and decompress data. See the “Compression” section in the module documentation. By default, this option is not present.
Prepared queries cache
Since Cassandra prepares queries on a per-node basis (and not on a per-connection basis), Xandra internally caches prepared queries for each connection or pool of connections. This means that if you prepare a query that was already prepared, no action will be executed on the Cassandra server and the prepared query will be returned from the cache.
If the Cassandra node goes down, however, the prepared query will be
invalidated and trying to use the one from cache will result in a
Xandra.Error
. However, this is automatically handled by Xandra: when such an
error is returned, Xandra will first retry to prepare the query and only
return an error if the preparation fails.
If you want to ensure a query is prepared on the server, you can set the
:force
option to true
.
Examples
{:ok, prepared} = Xandra.prepare(conn, "SELECT * FROM users WHERE id = ?")
{:ok, _page} = Xandra.execute(conn, prepared, [_id = 1])
{:error, %Xandra.Error{reason: :invalid_syntax}} = Xandra.prepare(conn, "bad syntax")
# Force a query to be prepared on the server and not be read from cache:
Xandra.prepare!(conn, "SELECT * FROM users WHERE ID = ?", force: true)
prepare!(conn, statement, Keyword.t) :: Xandra.Prepared.t | no_return
Prepares the given query, raising if there’s an error.
This function works exactly like prepare/3
, except it returns the prepared
query directly if preparation succeeds, otherwise raises the returned error.
Examples
prepared = Xandra.prepare!(conn, "SELECT * FROM users WHERE id = ?")
{:ok, _page} = Xandra.execute(conn, prepared, [_id = 1])
Acquires a locked connection from conn
and executes fun
passing such
connection as the argument.
All options are forwarded to DBConnection.run/3
(and thus some of them to
the underlying pool).
The return value of this function is the return value of fun
.
Examples
Preparing a query and executing it on the same connection:
Xandra.run(conn, fn conn ->
prepared = Xandra.prepare!(conn, "INSERT INTO users (name, age) VALUES (:name, :age)")
Xandra.execute!(conn, prepared, %{"name" => "John", "age" => 84})
end)
Starts a new connection or pool of connections to Cassandra.
This function starts a new connection or pool of connections to the provided
Cassandra server. options
is a list of both Xandra-specific options, as well
as DBConnection
options.
Options
These are the Xandra-specific options supported by this function:
:nodes
- (list of strings) the Cassandra nodes to connect to. Each node in the list has to be in the form"ADDRESS:PORT"
or in the form"ADDRESS"
: if the latter is used, the default port (9042
) will be used for that node. Defaults to["127.0.0.1"]
. This option must contain only one node unless the:pool
option is set toXandra.Cluster
; see the documentation forXandra.Cluster
for more information.:compressor
- (module) the compressor module to use for compressing and decompressing data. See the “Compression” section in the module documentation. By default this option is not present.:authentication
- (tuple) a two-element tuple: the authenticator module to use for authentication and its supported options. See the “Authentication” section in the module documentation.
The rest of the options are forwarded to DBConnection.start_link/2
. For
example, to start a pool of connections to Cassandra, the :pool
option can
be used:
Xandra.start_link(pool: DBConnection.Poolboy)
Note that this requires the poolboy
dependency to be specified in your
application. The following options have default values that are different from
the default values provided by DBConnection
:
:idle_timeout
- defaults to30_000
(30 seconds)
Examples
# Start a connection:
{:ok, conn} = Xandra.start_link()
# Start a connection and register it under a name:
{:ok, _conn} = Xandra.start_link(name: :xandra)
# Start a named pool of connections:
{:ok, _pool} = Xandra.start_link(name: :xandra_pool, pool: DBConnection.Poolboy)
As the DBConnection
documentation states, if using a pool it’s necessary to
pass a :pool
option with the pool module being used to every call. For
example:
{:ok, _pool} = Xandra.start_link(name: :xandra_pool, pool: DBConnection.Poolboy)
Xandra.execute!(:xandra_pool, "SELECT * FROM users", _params = [], pool: DBConnection.Poolboy)
Using a keyspace for new connections
It is common to start a Xandra connection or pool of connections that will use a single keyspace for their whole life span. Doing something like:
{:ok, conn} = Xandra.start_link()
Xandra.execute!(conn, "USE my_keyspace")
will work just fine when you only have one connection. If you have a pool of
connections (with the :pool
option), however, the code above won’t work:
that code would start the pool, and then checkout one connection from the pool
to execute the USE my_keyspace
query. That specific connection will then be
using the my_keyspace
keyspace, but all other connections in the pool will
not. Fortunately, DBConnection
provides an option we can use to solve this
problem: :after_connect
. This option can specify a function that will be run
after each single connection to Cassandra. This function will take a
connection and can be used to setup that connection; since this function is
run for every established connection, it will work well with pools as well.
{:ok, conn} = Xandra.start_link(after_connect: fn(conn) -> Xandra.execute(conn, "USE my_keyspace") end)
See the documentation for DBConnection.start_link/2
for more information
about this option.
stream_pages!(conn, statement | Xandra.Prepared.t, values, Keyword.t) :: Enumerable.t
Streams the results of a simple query or a prepared query with the given params
.
This function can be used to stream the results of query
so as not to load
them entirely in memory. This function doesn’t send any query to Cassandra
right away: it will only execute queries as necessary when results are
requested out of the returned stream.
The returned value is a stream of Xandra.Page
structs, where each of such
structs contains at most as many rows as specified by the :page_size
option. Every time an element is requested from the stream, query
will be
executed with params
to get that result.
In order to get each result from Cassandra, execute!/4
is used: this means
that if there is an error (such as a network error) when executing the
queries, that error will be raised.
Simple or prepared queries
Regardless of query
being a simple query or a prepared query, this function
will execute it every time a result is needed from the returned stream. For
this reason, it is usually a good idea to use prepared queries when streaming.
Options
options
supports all the options supported by execute/4
, with the same
default values.
Examples
prepared = Xandra.prepare!(conn, "SELECT * FROM users")
users_stream = Xandra.stream_pages!(conn, prepared, _params = [], page_size: 2)
[%Xandra.Page{} = _page1, %Xandra.Page{} = _page2] = Enum.take(users_stream, 2)