View Source Spannex.Protocol (spannex v0.5.3)

A DBConnection implementation for Cloud Spanner.

Connections

Each connection maintains its own gRPC channel to the Spanner API and its own unique session. The session is used as a unique identifier for all other requests made against the database.

Transactions

When starting a transaction, there are four accepted types of transactions:

  1. :batch_write - Uses "batch write" and "mutation groups" to execute multiple operations in a single transaction, optimized for "blind bulk writes", low-latency, and atomicity within each group of mutations.
  2. :read_write - A locking read-write transaction, this is what you would consider a "traditional transaction". It's best used when you are changing one row or a handful of related rows or need to reference data created in the same transaction.
  3. :read_only - A special transaction type that allows for point-in-time consistency in the data being read. This helps eliminate possible drift in values when operations might be happening in parallel to the read-only transaction by locking the state of the data being read to a specific point in time.
  4. :partitioned_dml - A special transaction where regular DML statements are partitioned and executed in parallel. This is useful for large-scale bulk operations that still need guarantees of atomicity and consistency.

The :batch_write transaction differs from :read_write and :partitioned_dml transactions in some important ways:

  1. It does not support reading data. It is strictly for writing data.
  2. You cannot reference changes made by other mutations within the same transaction.
  3. You must specify all literals explicitly, you cannot use named query parameters.
  4. All queries are locally queued and then sent all at once when the transaction is committed. This means individual statements are not sent to the database, rather they are batched into a single gRPC request when the transaction is comitted.
  5. Deletions must be done by primary key. If you need a custom where-clause to delete rows, you must do so in a :read_write or :partitioned_dml transaction using regular DML statements.

Timeouts

Transactions have a 10-second idle timeout that cannot be modified. If a transaction remains idle for more than 10 seconds, it is aborted. Aborted transactions cause the session to enter a limbo state where you can't really do anything useful anymore, so the default behavior is to disconnect and force the creation of a new connection.

With the exception of a :batch_write transactions, all individual statements resolved via handle_execute will reset the 10-second idle timer. But if you start a :batch_write transaction, you have 10 seconds to queue up all necessary statements and then commit the transaction. While this should be sufficient in most scenarios, it is worth keeping this in mind if you are using :batch_write transactions for very large operations. In such cases where the 10-second idle timeout might be a problem, consider pre-calculating as much data for the DML statements as possible and then starting a :batch_write transaction after.

Queries

Spanner requires that all write operations, i.e. queries that are not SELECT statements, use an explicit transaction. If you attempt to execute a write statement outside of a transaction, you will receive an error.

For read queries, i.e. SELECT statements, a temporary read-only transaction with a strong consistency guarantee can be created for you.

Spanner also stores version history for all columns and all rows and all tables, meaning you can get truly strong consistency guarantees when reading data. You can ensure that, given a specific (recent) point in time, all data read from the database is no more recent than that point in time.

Summary

Functions

Start a new gRPC connection to the Spanner API and also generate a new session.

Converts a single column value from its gRPC-encoded form to its Elixir-native representation. The type parameter is the native type of the column and the enc_type parameter is the type of the encoded value.

Converts the row data from a Google.Spanner.V1.ResultSet into a list of maps, where each map is one row of the result set. The keys of the map are the column names and the values are the column values. The column values are converted from their encoded form to their Elixir-native representation.

Begins a new transaction. You must use a transaction for any mutations, i.e. anything that isn't a SELECT query.

Commits a running transaction.

Would normally deallocate a cursor generated by handle_declare/4, but we don't support cursors in this implementation.

Would normally generate a cursor for a query prepared by handle_prepare/3, but we don't support cursors in this implementation.

Executes a SQL statement against the Spanner API.

Would normally fetch the next result from a cursor generated by handle_declare/4, but we don't support cursors in this implementation.

Rolls back a running transaction.

Creates a mutation struct for the given operation. For anything other than a :delete operation, the columns and rows parameters are used to create a Google.Spanner.V1.Mutation.Write struct.

Attempts to perform an inexpensive database operation to ensure the session is still valid. If successful, the operation should also ensure Spanner doesn't drop the session since it's technically an "active" session.

Parses a SQL query with optional named parameters and converts the operation to a mutation using make_mutation/4.

Types

@type t() :: %Spannex.Protocol{
  channel: GRPC.Channel.t(),
  goth: term(),
  queries: [DBConnection.query()],
  seqno: non_neg_integer(),
  session: Google.Spanner.V1.Session.t(),
  status: DBConnection.status(),
  transaction: transaction_type(),
  transaction_id: String.t()
}
@type transaction_type() ::
  :none | :batch_write | :read_write | :read_only | :partitioned_dml

Functions

No-op

Start a new gRPC connection to the Spanner API and also generate a new session.

There is only one required option for the connect/1 function: :database. This tells Spannex which database the session will be for.

If you do not supply an authorization header as part of :grpc_opts, you must also supply the :goth option, which tells Spannex how to fetch an access token.

You may optionally supply the following other options:

  • :goth - The term used to access the Goth library to fetch an access token.
  • :labels - A map of labels to apply to the session. These labels can be used to filter and monitor sessions using the Cloud console or admin API.
  • :host - The gRPC endpoint to connect to. If not supplied, the global Spanner API endpoint is used.
  • :grpc_opts - The options to pass to GRPC.Stub.connect/2.
  • :cred - A gRPC credential wrapper. If not supplied, the Goth library is used to fetch a Google Cloud access token.
  • :headers - A list of headers to use for the connection. If content-type and authorization are supplied, they will be overwritten.
Link to this function

convert_value(arg1, arg2, value)

View Source

Converts a single column value from its gRPC-encoded form to its Elixir-native representation. The type parameter is the native type of the column and the enc_type parameter is the type of the encoded value.

Link to this function

decode_results(result_set)

View Source

Converts the row data from a Google.Spanner.V1.ResultSet into a list of maps, where each map is one row of the result set. The keys of the map are the column names and the values are the column values. The column values are converted from their encoded form to their Elixir-native representation.

Link to this function

decode_row(list1, list2, acc)

View Source
Link to this function

handle_begin(opts, state)

View Source

Begins a new transaction. You must use a transaction for any mutations, i.e. anything that isn't a SELECT query.

May specify the option :type as any one of: :batch_write, :read_write, :read_only, :partitioned_dml. If :type is not set, it defaults to :read_write.

If :batch_write is specified, all subsequent handle_execute/4 calls will simply queue up the statements locally in a list that will be converted to a set of mutations when handle_commit/2 is called. If handle_rollback/2 is called instead, the queue is simply cleared, nothing happens remotely.

Link to this function

handle_close(query, opts, state)

View Source

No-op

Link to this function

handle_commit(opts, state)

View Source

Commits a running transaction.

If the transaction is a :batch_write transaction, all queued statements are converted to "mutations" and sent to the Spanner API in a single request.

Link to this function

handle_deallocate(query, cursor, opts, state)

View Source

Would normally deallocate a cursor generated by handle_declare/4, but we don't support cursors in this implementation.

Link to this function

handle_declare(query, params, opts, state)

View Source

Would normally generate a cursor for a query prepared by handle_prepare/3, but we don't support cursors in this implementation.

Link to this function

handle_execute(query, params, opts, state)

View Source

Executes a SQL statement against the Spanner API.

All statements that are mutations (i.e. not SELECT statements) must be executed within a transaction or they will produce an error.

If a SELECT statement is executed outside of a transaction, the database will generate a temporary :read_only transaction to run it in.

If the transaction type is :batch_write, the query will be queued up locally and not sent to the Spanner API until handle_commit/2 is called. This means the result portion of the response tuple will be nil since there's no result to return yet.

Link to this function

handle_fetch(query, cursor, opts, state)

View Source

Would normally fetch the next result from a cursor generated by handle_declare/4, but we don't support cursors in this implementation.

Link to this function

handle_prepare(query, opts, state)

View Source

No-op

Link to this function

handle_rollback(opts, state)

View Source

Rolls back a running transaction.

If the transaction is a :batch_write transaction, all queued statements are simply cleared and the state is returned to idle.

Link to this function

make_mutation(operation, table, columns, rows)

View Source

Creates a mutation struct for the given operation. For anything other than a :delete operation, the columns and rows parameters are used to create a Google.Spanner.V1.Mutation.Write struct.

For a :delete operation, the Spanner API expects that the primary key will be used to execute the delete mutation. If you need to delete using a where-clause and not the primary key, you cannot use a mutation. The rows parameter is expected to be a list of values that correspond to the primary key columns in the same order as they are defined in the table schema.

Attempts to perform an inexpensive database operation to ensure the session is still valid. If successful, the operation should also ensure Spanner doesn't drop the session since it's technically an "active" session.

Parses a SQL query with optional named parameters and converts the operation to a mutation using make_mutation/4.