Roux.Runtime (roux v0.2.2)

Copy Markdown View Source

The query execution engine.

Handles memoization, dependency tracking, cycle detection, validation, early cutoff, dedup, and write buffering. This is the core integration point where queries, memos, validation, and entities come together.

Execution flow

execute/4 is the main entry point, called by defquery-generated functions. It checks the memo table, validates stale entries, and re-executes when needed. Results are buffered during execution and flushed to ETS on completion.

Concurrency

execute/4 is synchronous and concurrent-safe. The dedup table prevents duplicate computation when multiple processes request the same query. Callers own their concurrency (e.g. Task.async_stream). query/3 executes inline. parallel/3 fans out, recording the fan-out as one dependency.

See D3, D13, D14 for design rationale.

Summary

Functions

The code version of the query whose body is running (Roux.Query), or nil for a query without one: for a body that keys something of its own — an action-cache entry, a file it keeps — on the code computing it.

Creates or updates an entity from within a query body.

Drops the values this process cached while serving db's queries.

Executes a query with memoization and dependency tracking.

Reads an entity field from within a query body.

Records that the value the running query returns names digests in the database's Roux.Blob store (files its body put there, say): while a manifest keeps the entry, it keeps them alive (Roux.Blob.retain/3). A value that only passes on digests an entry it read already holds need not hold them again — unless that entry is not kept (store: :none, transient:).

Reads an input value from within a query body.

Reads an input value, or default when the key has none.

Reads an input value, short-circuiting on missing keys.

Looks up an entity by identity key from within a query body.

Demands independent queries concurrently, and returns their values in the order given.

Calls a derived query from within a query body.

Calls a derived query, short-circuiting on errors.

Reads all fields from an entity as a map.

Records that the current query depends on another query.

Runs fun with dependency recording and durability propagation suppressed for the ENCLOSING query.

Functions

code_version()

@spec code_version() :: binary() | nil

The code version of the query whose body is running (Roux.Query), or nil for a query without one: for a body that keys something of its own — an action-cache entry, a file it keeps — on the code computing it.

Raises ArgumentError outside a query body.

create(db, module, attrs)

@spec create(Roux.Database.t(), module(), map()) :: Roux.Entity.entity_id()

Creates or updates an entity from within a query body.

Delegates to Roux.Entity.create/4 with the current revision and records the entity in the context's created_entities for GC tracking. Returns the entity ID.

drop_cached_values(database)

@spec drop_cached_values(Roux.Database.t()) :: :ok

Drops the values this process cached while serving db's queries.

A process that serves queries keeps every value it served on its own heap until the key is recomputed or the process exits. A long-lived process that shuts a database down should call this alongside Roux.Database.shutdown/1.

execute(db, query_name, key, query_fun)

@spec execute(Roux.Database.t(), atom(), term(), (Roux.Database.t(), term() -> term())) ::
  term()

Executes a query with memoization and dependency tracking.

Called by defquery-generated functions. Checks the memo table for a cached result, validates stale entries, and re-executes when needed.

The query_fun receives (db, key) and returns the query result.

field(db, module, entity_id, field_name)

@spec field(Roux.Database.t(), module(), Roux.Entity.entity_id(), atom()) :: term()

Reads an entity field from within a query body.

Delegates to Roux.Entity.field/4 and records a field-level dependency so the query is invalidated only when that specific field changes.

hold(digests)

@spec hold(Roux.Blob.digest() | [Roux.Blob.digest()]) :: :ok

Records that the value the running query returns names digests in the database's Roux.Blob store (files its body put there, say): while a manifest keeps the entry, it keeps them alive (Roux.Blob.retain/3). A value that only passes on digests an entry it read already holds need not hold them again — unless that entry is not kept (store: :none, transient:).

Raises ArgumentError outside a query body.

input(db, input_name, key)

@spec input(Roux.Database.t(), atom(), term()) :: term()

Reads an input value from within a query body.

Records a dependency on the input and tracks its durability level in the current context for the durability optimization.

input(db, input_name, key, opts)

@spec input(Roux.Database.t(), atom(), term(), keyword()) :: term()

Reads an input value, or default when the key has none.

An unset key is a dependency like a set one: the reader records that the input was absent, and setting it later invalidates the reader (Roux.Validation). While it stays unset the reader validates as fresh — where reading Roux.Input.exists?/3 first and depending on the input only when it is set would leave a reader that never learns it was set, and depending on an unset input always would re-run the reader on every validation.

Options

input!(db, input_name, key)

@spec input!(Roux.Database.t(), atom(), term()) :: term()

Reads an input value, short-circuiting on missing keys.

Like input/3, but if the key has not been set, throws a {:roux_query_error, reason} that is automatically caught by the enclosing defquery and converted to {:error, reason}.

lookup(db, module, identity_key)

@spec lookup(Roux.Database.t(), module(), tuple()) ::
  {:ok, Roux.Entity.entity_id()} | :error

Looks up an entity by identity key from within a query body.

Non-interning lookup — does not record a dependency since the identity mapping is structural, not a data dependency.

parallel(db, queries, opts \\ [])

@spec parallel(Roux.Database.t(), [{atom(), term()}], keyword()) :: [term()]

Demands independent queries concurrently, and returns their values in the order given.

Inside a query body, the enclosing query records the whole fan-out as ONE dependency, {:parallel, max_concurrency, keys}, and validation brings its members up to date concurrently too, then checks each for a change — where a dependency per member would validate them one by one, re-executing stale ones in turn. The members run in processes linked to the caller, so cancelling the caller (Roux.Cancellation) takes them down with it; they carry the caller's query stack, so a cycle through them is detected. What a member raises is raised again here.

Options

  • :max_concurrency — how many members run at once (default System.schedulers_online/0), for execution and validation alike;
  • :timeout — how long to wait for each member, :infinity by default: a member's own work bounds itself.

query(db, query_name, key)

@spec query(Roux.Database.t(), atom(), term()) :: term()

Calls a derived query from within a query body.

Records a dependency on the called query and dispatches to the registered query function. Executes inline (same process).

query!(db, query_name, key)

@spec query!(Roux.Database.t(), atom(), term()) :: term()

Calls a derived query, short-circuiting on errors.

Like query/3, but if the result matches {:error, reason}, throws a {:roux_query_error, reason} that is automatically caught by the enclosing defquery and converted back to {:error, reason}.

This eliminates nested case statements for error propagation:

# Instead of:
case Roux.Runtime.query(db, :typecheck, uri) do
  {:ok, types} -> use(types)
  {:error, _} = err -> err
end

# Write:
{:ok, types} = Roux.Runtime.query!(db, :typecheck, uri)
use(types)

read(db, module, entity_id)

@spec read(Roux.Database.t(), module(), Roux.Entity.entity_id()) :: map()

Reads all fields from an entity as a map.

Calls field/4 for each field, so a field-level dependency is recorded for every field. Use this when the consumer needs the whole entity; use field/4 when it only needs a subset.

record_dependency(ctx, query_key)

Records that the current query depends on another query.

Pure function that returns an updated context. Called internally by query/3 and input/3 via the process-dictionary helper.

untracked(fun)

@spec untracked((-> result)) :: result when result: var

Runs fun with dependency recording and durability propagation suppressed for the ENCLOSING query.

Nested queries inside the block still execute normally — memoized, deduplicated, and cycle-checked in the same process — and record their own dependencies in their own memo entries. Only the caller's edges are discarded.

For demand-driven warm-up whose exact dependencies are recorded separately: a compiler pre-loading hinted modules before compiling, with precise edges recorded from a tracer afterward. The hint list over-approximates, so tracking it would over-invalidate.

By design the enclosing query does NOT re-run when an untracked-only input changes — that is the whole point, and it means the caller is responsible for recording the real edges some other way.

The query stack is deliberately preserved, so a cycle through an untracked call still raises Roux.Cycle.Error.