AshScylla Implementation Summary

Technical overview of the AshScylla data layer implementation

Overview

AshScylla is a comprehensive data layer for the Ash Framework that enables persistence with ScyllaDB or Apache Cassandra. It uses Xandra (a native Elixir CQL driver) to communicate via CQL (Cassandra Query Language).

Current version: 0.13.1

Architecture

┌─────────────────────────────────────────────────────────┐
│                    Ash Framework                        │
│  (Resources, Actions, Queries, Filters)               │
└──────────────────┬──────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────────────────┐
│              AshScylla.DataLayer                       │
│  • Implements Ash.DataLayer behaviour                  │
│  • Converts Ash queries to CQL                        │
│  • Handles CRUD operations                            │
└──────────────────┬──────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────────────────┐
│                  AshScylla.Query                        │
│  • Owns the query struct                              │
└──────────────────┬──────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────────────────┐
│                  AshScylla.QueryBuilder                │
│  • Converts Ash filters to CQL WHERE clauses          │
│  • Builds optimized CQL queries                       │
└──────────────────┬──────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────────────────┐
│                    Xandra (direct)                      │
│  • Native Elixir CQL driver               │
└──────────────────┬──────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────────────────┐
│                   ScyllaDB/Cassandra                   │
└─────────────────────────────────────────────────────────┘

Files Structure

Core Implementation

Data Layer Modules

Mix Tasks

Test Files

Features Implemented

Core Ash.DataLayer Features

Features NOT Supported

ScyllaDB-Specific Features

1. TTL (Time To Live)

scylla do
  ttl 3600  # Expire after 1 hour
end

• TTL applied to INSERT statements via USING TTL clause

2. Consistency Levels

scylla do
  consistency :quorum  # :any, :one, :two, :three, :quorum, :all, :local_quorum
end

• Supports all ScyllaDB consistency levels

3. Secondary Indexes

scylla do
  secondary_index :email              # Single column
  secondary_index [:name, :age]        # Composite index (multi-column)
  secondary_index :status, name: "idx_status"
end

• ScyllaDB OSS doesn't support multi-column secondary indexes — generates separate single-column indexes

4. Materialized Views

scylla do
    primary_key: [:email, :id],
    include_columns: [:name, :age],
    clustering_order: [id: :desc]
end

• Automatic CQL generation for CREATE MATERIALIZED VIEW

5. Batch Operations

# Synchronous batch
AshScylla.DataLayer.Batch.batch_insert(repo, statements)

# Async partition-aware batching
AshScylla.DataLayer.Batch.batch_insert_async(repo, statements, max_concurrency: 8)

• Supports BATCH INSERT, UPDATE, DELETE
• Async mode groups by partition key for safety

6. Token-Based Pagination

# First page
{:ok, records, next_token} =
  AshScylla.DataLayer.Pagination.fetch_page(repo, table, filters, nil, 10)

# Subsequent pages
{:ok, records, next_token} =
  AshScylla.DataLayer.Pagination.fetch_page(repo, table, filters, next_token, 10)

• Uses Xandra's native paging_state mechanism
• Default page size: 50, max: 1000

7. Prepared Statement Caching

children = [
  AshScylla.PreparedStatementCache,
  # ...
]

• GenServer + ETS cache
• Automatic prepared statement reuse
• Max 10,000 entries, cleanup every 5 minutes

8. Per-Action Consistency

scylla do
  consistency :quorum
  per_action_consistency read: :one, create: :quorum
end

9. Lightweight Transactions (LWT)

scylla do
  lwt true
end

• Enables IF NOT EXISTS on create, IF clauses on update

10. Query Optimization

• Filter validation prevents ALLOW FILTERING anti-pattern
• In-memory sort compensation when ORDER BY is dropped due to secondary index scan
• Query optimizer hints (consistency, timeout, paging, speculative retry)

11. Application-Level Compression

• Supports LZ4, Snappy, Deflate, Zstd
• Table-level compression CQL generation
• Transparent field-level compression

12. User Defined Types (UDT)

• Full encoding/decoding for Xandra
• CQL generation for CREATE/ALTER/DROP TYPE

13. Collection Types

• LIST, SET, MAP encoding for Xandra
• CONTAINS/CONTAINS KEY filter support
• Frozen collection support

Data Layer Query Struct

# AshScylla.Query struct (lib/ash_scylla/query.ex)
defstruct [
  :resource,
  :repo,
  :table,
  limit: nil,
  select: nil,
  distinct: nil,
  tenant: nil,
  context: %{},
  atomic: nil,
  upsert?: false,
  upsert_fields: [],
  upsert_identity: nil,
  keyset: nil,
  aggregates: [],
  group_by: nil,
  filters: [],
  sorts: []
]

Error Handling

AshScylla provides structured error handling for ScyllaDB-specific errors:

Error Types

Using Error Handling

case AshScylla.DataLayer.run_query(query, resource) do
  {:ok, results} ->
    {:ok, results}

  {:error, %AshScylla.Error.ScyllaError{} = error} ->
    Logger.error("Database error: #{AshScylla.Error.format_error(error)}")

    if AshScylla.Error.retryable?(error) do
      {:retry, error}
    else
      {:error, error}
    end
end

Dependencies

Dev/test dependencies:

Current Status

Working Features

• Compiles successfully with no errors
• 1000+ unit tests across 19 feature domains
• Integration tests with real ScyllaDB (via Podman/testcontainers)
• Full CRUD operations with TTL, consistency, LWT
• Secondary indexes, materialized views, UDTs
• Batch operations (sync + async partition-aware)
• Token-based pagination
• Prepared statement caching
• Comprehensive error handling with retry logic
• Telemetry integration
• Schema migration system (AshScylla.Schema)
• Resource template generation
• Compression and collection type support
• Aggregate support: COUNT, SUM, AVG, MIN, MAX (query + relationship aggregates)

Not Supported (ScyllaDB Limitations)

• JOINs (use denormalization)
• Complex aggregations across partitions (only per-partition COUNT)
• ACID transactions across partitions (only lightweight transactions)
• ALLOW FILTERING (rejected at query-plan time — add secondary indexes instead)
• OR conditions in WHERE clause (rewritten to IN where possible)
• Foreign keys
• has_many / many_to_many relationship aggregates (use denormalization or materialized views)
• :first, :list, :exists, :custom aggregate kinds

Running Tests

Unit Tests

mix test --exclude integration

Integration Tests

# With Podman container (default)
mix test --only integration

# Against local ScyllaDB
SCYLLA_DIRECT=1 mix test --only integration

# Cluster tests (3-node, requires Podman)
mix test test/integration/cluster_integration_test.exs --only integration

# Specific test file
mix test test/integration/scylla_integration_test.exs
mix test test/unit/data_layer/data_layer_crud_test.exs

Coverage

mix test --exclude integration --cover

Generates cover/index.html.

Example Usage

# Configure Repo
defmodule MyApp.Repo do
  use AshScylla.Repo,
    otp_app: :my_app
end

# Configure Resource with all features
defmodule MyApp.User do
  use Ash.Resource,
    data_layer: AshScylla.DataLayer,
    domain: MyApp.Domain


  scylla do
    table "users"
    consistency :quorum
    ttl 3600
    lwt true

    secondary_index :email

    materialized_view :users_by_email,
      primary_key: [:email, :id],
      include_columns: [:name, :age]
  end

  attributes do
    uuid_primary_key :id
    attribute :name, :string
    attribute :email, :string
    attribute :age, :integer
  end

  actions do
    defaults [:create, :read, :update, :destroy]
  end
end

License

Apache License 2.0 - see LICENSE file for details.