# Kura

Kura

Database layer for Erlang - Ecto-equivalent abstractions in pure Erlang. Pluggable backends: [`kura_postgres`](https://github.com/Taure/kura_postgres) (PostgreSQL via pgo), [`kura_sqlite`](https://github.com/Taure/kura_sqlite) (SQLite via esqlite). ## Features - **Schema** - behaviour-based schema definitions with type metadata - **Changeset** - cast external params, validate, track changes and errors - **Query Builder** - composable, functional query construction - **SQL Compiler** - parameterized SQL generation (no string interpolation) - **Repo** - CRUD operations with automatic type conversion and PG error mapping - **Associations** - `belongs_to`, `has_one`, `has_many`, `many_to_many` with preloading - **Composite keys** - N-column primary keys (`key/0`) and composite foreign keys (`#kura_ref{}`) - **Embedded Schemas** - `embeds_one`, `embeds_many` stored as JSONB - **Multi** - atomic transaction pipelines - **Migrations** - DDL operations with automatic module-based discovery - **Enums** - atom-backed enum types stored as `VARCHAR` - **Telemetry** - query logging with timing - **Lifecycle Hooks** - before/after callbacks for insert, update, delete - **Audit Trail** - automatic change tracking with actor context - **Pagination** - offset-based and cursor-based pagination - **Streaming** - server-side cursor streaming for large result sets - **Multitenancy** - schema prefix and attribute-based tenant isolation - **Optimistic Locking** - concurrent update conflict detection - **Encryption at rest** - `{encrypted, Type}` fields, AES-256-GCM, key rotation ## Quick Start ### Define a Schema ```erlang -module(user). -behaviour(kura_schema). -include_lib("kura/include/kura.hrl"). -export([table/0, fields/0]). table() -> ~"users". fields() -> [ #kura_field{name = id, type = id, primary_key = true, nullable = false}, #kura_field{name = name, type = string, nullable = false}, #kura_field{name = email, type = string, nullable = false}, #kura_field{name = age, type = integer}, #kura_field{name = inserted_at, type = utc_datetime}, #kura_field{name = updated_at, type = utc_datetime} ]. ``` ### Define a Repo ```erlang -module(my_repo). -behaviour(kura_repo). -export([otp_app/0, start/0, all/1, get/2, insert/1, update/1, delete/1]). otp_app() -> my_app. start() -> kura_repo_worker:start(?MODULE). all(Q) -> kura_repo_worker:all(?MODULE, Q). get(Schema, Id) -> kura_repo_worker:get(?MODULE, Schema, Id). insert(CS) -> kura_repo_worker:insert(?MODULE, CS). update(CS) -> kura_repo_worker:update(?MODULE, CS). delete(CS) -> kura_repo_worker:delete(?MODULE, CS). ``` Configure the database connection in `sys.config`: ```erlang [{my_app, [ {my_repo, #{ database => ~"myapp", hostname => ~"localhost", port => 5432, username => ~"postgres", password => <<>>, pool_size => 10 }} ]}]. ``` ### Changesets ```erlang %% Cast and validate external params CS = kura_changeset:cast(user, #{}, #{~"name" => ~"Alice", ~"email" => ~"alice@example.com"}, [name, email, age]), CS1 = kura_changeset:validate_required(CS, [name, email]), CS2 = kura_changeset:validate_format(CS1, email, ~"@"), CS3 = kura_changeset:validate_length(CS2, name, [{min, 1}, {max, 100}]), %% Insert {ok, User} = my_repo:insert(CS3). ``` ### Query Builder ```erlang Q = kura_query:from(user), Q1 = kura_query:where(Q, {age, '>', 18}), Q2 = kura_query:where(Q1, {'or', [{role, ~"admin"}, {role, ~"moderator"}]}), Q3 = kura_query:select(Q2, [name, email]), Q4 = kura_query:order_by(Q3, [{name, asc}]), Q5 = kura_query:limit(Q4, 10), {ok, Users} = my_repo:all(Q5). ``` Supported conditions: `=`, `!=`, `<`, `>`, `<=`, `>=`, `like`, `ilike`, `matches`, `in`, `not_in`, `is_nil`, `is_not_nil`, `between`, `{'and', [...]}`, `{'or', [...]}`, `{'not', ...}`, `{fragment, SQL, Params}`. #### Full-text search `{Field, matches, Query}` is a full-text search: on PostgreSQL it compiles to `to_tsvector(Field) @@ plainto_tsquery($n)`, with the query text bound as a parameter (`plainto_tsquery` treats it as plain words, so no query syntax is needed or trusted). Requires a backend declaring the `full_text_search` capability. ```erlang Q = kura_query:where(kura_query:from(article), {body, matches, ~"erlang database"}), {ok, Hits} = my_repo:all(Q). ``` Matching uses the server's `default_text_search_config` for stemming and stop-words. For production, add a GIN expression index so the search doesn't recompute a `tsvector` per row: ```sql CREATE INDEX articles_body_fts ON articles USING gin (to_tsvector('english', body)); ``` #### Window functions Use `over/2` inside `select_expr/2` for `OVER (PARTITION BY ... ORDER BY ...)` expressions. The window function is an aggregate (`{count, '*'}`, `{sum, Field}`, `{avg, Field}`, `{min, Field}`, `{max, Field}`) or a ranking function (`row_number`, `rank`, `dense_rank`). ```erlang Q = kura_query:select_expr(kura_query:from(sales), [ {category, category}, {row_num, kura_query:over(row_number, #{partition_by => [category], order_by => [{amount, desc}]})}, {running_total, kura_query:over({sum, amount}, #{partition_by => [category], order_by => [{day, asc}]})} ]), {ok, Rows} = my_repo:all(Q). ``` Requires a backend declaring the `window_functions` capability — PostgreSQL today (SQLite window rendering is planned). ### Migrations ```erlang -module(m20240115120000_create_users). -behaviour(kura_migration). -include_lib("kura/include/kura.hrl"). -export([up/0, down/0]). up() -> [{create_table, ~"users", [ #kura_column{name = id, type = id, primary_key = true, nullable = false}, #kura_column{name = name, type = string, nullable = false}, #kura_column{name = email, type = string, nullable = false}, #kura_column{name = age, type = integer}, #kura_column{name = inserted_at, type = utc_datetime}, #kura_column{name = updated_at, type = utc_datetime} ]}, {create_index, ~"users", [email], #{unique => true}}]. down() -> [{drop_index, ~"users_email_index"}, {drop_table, ~"users"}]. ``` Run migrations: ```erlang kura_migrator:migrate(my_repo). kura_migrator:rollback(my_repo). kura_migrator:status(my_repo). ``` ## Type Mapping | Kura | PostgreSQL | SQLite | Erlang | |---|---|---|---| | `id` | `BIGSERIAL` | `INTEGER PRIMARY KEY` | `integer()` | | `integer` | `INTEGER` | `INTEGER` | `integer()` | | `float` | `DOUBLE PRECISION` | `REAL` | `float()` | | `string` | `VARCHAR(255)` | `TEXT` | `binary()` | | `text` | `TEXT` | `TEXT` | `binary()` | | `boolean` | `BOOLEAN` | `INTEGER` (0/1) | `boolean()` | | `date` | `DATE` | `TEXT` (ISO 8601) | `{Y, M, D}` | | `utc_datetime` | `TIMESTAMPTZ` | `TEXT` (ISO 8601) | `calendar:datetime()` | | `uuid` | `UUID` | `TEXT` | `binary()` | | `jsonb` | `JSONB` | `TEXT` | `map()` | | `{array, T}` | `T[]` | unsupported | `list()` | | `{encrypted, T}` | `BYTEA` | `BLOB` | `T` (encrypted at rest) | | `{vector, N}` | `VECTOR(N)` | unsupported | `[float()]` | SQLite values round-trip transparently via `kura_types:cast/2` (booleans 0/1 → `true`/`false`, ISO 8601 → datetime tuples, JSON text → maps). `{vector, N}` (and dimensionless `vector`) map to [pgvector](https://github.com/pgvector/pgvector) columns for embeddings; requires `CREATE EXTENSION vector` and a backend declaring the `vector` capability. A value is a list of numbers, dimension-checked on cast. Distance-query operators (`<->` / `<=>`) are not yet a first-class query form - use a `{fragment, SQL, Params}` for nearest-neighbour ordering meanwhile. Elements are stored as `float4`, so a value read back from a column is the single-precision approximation of what was written, not the exact double. ## Encryption at rest Wrap a field type in `{encrypted, Type}` to store it AES-256-GCM-encrypted in a `BYTEA` column. Values are encrypted on write and decrypted on read; casting and validation still run on the plaintext. ```erlang fields() -> [ #kura_field{name = id, type = id, primary_key = true}, #kura_field{name = email, type = string}, #kura_field{name = ssn, type = {encrypted, string}, nullable = false} ]. ``` Supported inner types: `string`, `text`, `binary`, `uuid`, `jsonb`, `integer`, `bigint`, `smallint`, `boolean`. Configure keys under the `kura` app env - each a base64-encoded 32-byte key, with `active` selecting the one used for new writes: ```erlang [{kura, [ {encryption, #{ active => 1, keys => [{1, <<"base64-encoded-32-byte-key">>}] }} ]}]. ``` **Rotation:** add a new key and move `active` to it. Each ciphertext carries its key id, so existing rows keep decrypting under their original key - no backfill. A custom keyring (KMS/Vault) can replace the default by implementing the `kura_keyring` behaviour and setting `{kura, [{keyring, my_keyring}]}`. **Constraints:** encrypted columns are **not queryable** and cannot carry a unique constraint (each write has a random nonce, so equal plaintexts produce different ciphertexts). Primary and foreign keys cannot be encrypted. `NULL` stays `NULL` (unencrypted), so presence still leaks. Schema introspection (`rebar3 kura gen_schemas`) cannot detect that a `BYTEA` column was encrypted - annotate it by hand. The audit trail redacts encrypted fields to `[encrypted]` (it keeps the "changed" signal but never logs the value). Ciphertext is not yet bound to its row/column, so a database-level actor could copy a ciphertext between rows of the same column; per-column binding is planned. ## Configuration Configure repos under the `kura` app env. Each repo is a map keyed by its module name; pick a backend package and Kura starts the configured pool at app boot, populating `dialect`, `pool_module`, and `driver_module` from the aggregator automatically. ```erlang %% sys.config — single Postgres repo [{kura, [ {repos, #{ my_repo => #{ backend => kura_backend_postgres, host => "localhost", port => 5432, database => "my_app_dev", user => "postgres", password => "postgres", pool_size => 10 } }} ]}]. ``` ```erlang %% sys.config — single SQLite repo [{kura, [ {repos, #{ my_repo => #{ backend => kura_backend_sqlite, database => <<"my_app.db">>, %% or <<":memory:">> pool_size => 4 } }} ]}]. ``` ```erlang %% sys.config — Postgres primary + SQLite analytics [{kura, [ {repos, #{ my_repo => #{ backend => kura_backend_postgres, host => "localhost", database => "main", user => "postgres", pool_size => 10 }, analytics_repo => #{ backend => kura_backend_sqlite, database => <<":memory:">> } }} ]}]. ``` Each repo module declares itself in code: ```erlang -module(my_repo). -behaviour(kura_repo). -export([otp_app/0]). otp_app() -> my_app. ``` Queries through `my_repo` emit Postgres SQL; queries through `analytics_repo` emit SQLite SQL. The query cache is keyed per repo so the dialects never share entries. UUID primary keys are auto-generated on insert when no value is provided. The default is **UUIDv4** (random) - secure by default, since a v4 key discloses nothing about the row. Set `uuid_version => v7` in a repo's config to use time-ordered UUIDv7 for better index locality; note that a v7 key embeds its creation timestamp, so avoid it for keys exposed in URLs or public APIs. A schema's own `generate_id/0` callback still takes precedence over both. ### Read replicas A replica is just another repo pointed at a read replica. Mark it `read_only => true` so writes are refused, and list it under the primary's `replicas` so the primary can hand it out: ```erlang {repos, #{ my_repo => #{backend => kura_backend_postgres, host => "primary", replicas => [my_repo_ro]}, my_repo_ro => #{backend => kura_backend_postgres, host => "replica", read_only => true} }} ``` `read_only => true` makes every write (insert/update/delete and the bulk and soft-delete variants) return `{error, read_only}`. Routing is explicit - pick a replica per read and own read-after-write yourself: ```erlang Replica = kura_repo:replica(my_repo), %% a replica repo, or my_repo if none {ok, Rows} = Replica:all(Q). ``` There is no automatic write/read split - reads go to whichever repo you call, so a read that must see a just-written row should use the primary. The guard covers the changeset write API; raw SQL via `query/3` is not guarded, so a raw write on a replica is rejected by PostgreSQL itself.
Legacy v1.x config forms (still supported) The flat single-repo form: ```erlang [{kura, [ {repo, my_repo}, {backend, kura_backend_postgres}, {host, "localhost"}, {port, 5432}, {database, "my_app_dev"}, {user, "postgres"}, {password, "postgres"}, {pool_size, 10} ]}]. ``` The per-app form: ```erlang [{my_app, [ {my_repo, #{ backend => kura_backend_postgres, database => ~"my_app_dev", hostname => ~"localhost", port => 5432, username => ~"postgres", password => ~"postgres", pool_size => 10 }} ]}]. ``` The per-app form requires the consuming app to call `my_repo:start()` manually. The `{repos, #{...}}` form (above) is preferred for new projects - single-repo today, no rewrite when you add a second backend.
Migrations are discovered automatically from compiled modules implementing the `kura_migration` behaviour. Optional telemetry/logging config: ```erlang [{kura, [ {log, true} %% true | {M, F} | false (default) ]}]. ``` ## Plugins - [rebar3_kura](https://github.com/Taure/rebar3_kura) - Rebar3 plugin that auto-generates migration files from schema changes. Add a field to your schema, run `rebar3 compile`, and the migration is created for you. - [opentelemetry_kura](https://github.com/Taure/opentelemetry_kura) - OpenTelemetry instrumentation. Subscribes to Kura's telemetry events and creates spans for every database query. ## Examples - [pet_store](https://github.com/Taure/pet_store) - A sample REST API built with Kura and Nova demonstrating schemas, changesets, queries, migrations, and associations in practice. ## Requirements - Erlang/OTP 28+ - One backend: PostgreSQL 14+ (via [kura_postgres](https://github.com/Taure/kura_postgres)) or SQLite 3.35+ (via [kura_sqlite](https://github.com/Taure/kura_sqlite))