Sublimate.ProjectionStrategy behaviour (Sublimate v0.1.0)

Copy Markdown

The contract a projection implements: what to derive from the source, and how to apply it to the destination table.

By implementing this behaviour, a consuming application or library defines a projection: a derivation that keeps a destination table synchronized with a source table as the source and its joined tables change. The strategy supplies everything specific to that projection - which tables its values are read from, the shape of the deltas table, the SQL each trigger writes, and the body of the merge that applies staged deltas to the destination.

For example, a strategy might derive a facet index - mapping each value to the set of source rows that have it - or a denormalized, searchable document table that flattens a source row together with its joined tables.

What a strategy defines

Beyond prepare/3 and chains/1, a strategy defines three things:

Chain primitives

Callbacks that must reach affected source rows across joins build their SQL around Sublimate.Projection's public chain primitives rather than walking the chain themselves. Which chain a callback needs depends on the leaf table a value comes from, so the primitives are invoked per value, from within the strategy.

Summary

Callbacks: deltas table shape

deltas_table_columns_sql(projection_data, prepared)

Column definitions for the deltas table - the part inside CREATE TABLE (...).

deltas_table_unique_index_columns(projection_data)

Columns forming the deltas table's unique index - what triggers' ON CONFLICT targets.

Callbacks: generation

chains(projection_data)

The chains this projection maintains - one per field whose value is read across joins.

prepare(projection_data, repo, postgrex_options)

Runs once at the start of installation. Fetches or computes whatever the later callbacks need, and validates the strategy's config against the database.

Callbacks: merge

merge_function_body(projection_data, prepared)

Body of the merge function: applies pending deltas into the destination table. Sublimate.Projection supplies the CREATE FUNCTION wrapper and the BEGIN/END.

Callbacks: trigger bodies

declarations(trigger_kind, projection_data, prepared)

DECLARE contents for a trigger function, or "" when the strategy needs no locals there. Sublimate.Projection omits the DECLARE block entirely when empty.

leaf_delete(projection_data, chain_spec, join, prepared)

Returns the SQL that records the removal of a joined (leaf) row's contribution when that row is deleted.

leaf_update(projection_data, chain_spec, join, prepared)

Body of a leaf table's AFTER UPDATE trigger function.

link_delete(projection_data, chain_spec, join, prepared)

Body of an intermediate table's BEFORE DELETE trigger function.

link_insert(projection_data, chain_spec, join, prepared)

Body of an intermediate table's TG_OP = 'INSERT' branch.

link_update(projection_data, chain_spec, join, prepared)

Body of an intermediate table's TG_OP = 'UPDATE' branch.

source_delete(projection_data, prepared)

Body of the source trigger's TG_OP = 'DELETE' branch.

source_insert(projection_data, prepared)

Body of the source trigger's TG_OP = 'INSERT' branch.

source_update(projection_data, prepared)

Body of the source trigger's TG_OP = 'UPDATE' branch.

Types

branch_sql()

SQL for one trigger branch. May contain several statements: a facet update emits a remove-old and an add-new. Sublimate.Projection splices it into the branch as-is, between THEN and the RETURN.

chain_spec()

One field's linear chain: the field it serves, the joins from the source to the table its value is read from, and that leaf table.

join_spec()
join_triggers()
prepared()

Opaque, strategy-specific data built by prepare/3.

trigger_kind()

Which trigger function's DECLARE block is being built.

Callbacks: deltas table shape

deltas_table_columns_sql(projection_data, prepared)

@callback deltas_table_columns_sql(projection_data :: struct(), prepared()) :: String.t()

Column definitions for the deltas table - the part inside CREATE TABLE (...).

deltas_table_unique_index_columns(projection_data)

@callback deltas_table_unique_index_columns(projection_data :: struct()) :: [String.t()]

Columns forming the deltas table's unique index - what triggers' ON CONFLICT targets.

Callbacks: generation

chains(projection_data)

@callback chains(projection_data :: struct()) :: [chain_spec()]

The chains this projection maintains - one per field whose value is read across joins.

The engine unions these into per-table triggers (see Sublimate.ChainUnion): a table appearing in several chains, possibly in different roles, gets a single trigger that serves all of them.

Fields whose value is on the source table need not appear here; the source trigger covers them. A strategy may include them with an empty chain or omit them.

Examples

def chains(strategy_data) do
  strategy_data.facets
  |> Enum.filter(&Map.get(&1, :join_table))
  |> Enum.map(fn facet ->
    %{
      field: facet.facet_name,
      joins: facet.joins,
      leaf_table: facet.join_table
    }
  end)
end

prepare(projection_data, repo, postgrex_options)

@callback prepare(
  projection_data :: struct(),
  repo :: module(),
  postgrex_options :: keyword()
) :: {:ok, prepared()} | {:error, term()} | {:error, atom(), term()}

Runs once at the start of installation. Fetches or computes whatever the later callbacks need, and validates the strategy's config against the database.

Returning an error tuple aborts installation and is passed through to the caller unchanged.

Callbacks: merge

merge_function_body(projection_data, prepared)

@callback merge_function_body(projection_data :: struct(), prepared()) :: String.t()

Body of the merge function: applies pending deltas into the destination table. Sublimate.Projection supplies the CREATE FUNCTION wrapper and the BEGIN/END.

Callbacks: trigger bodies

declarations(trigger_kind, projection_data, prepared)

@callback declarations(
  trigger_kind(),
  projection_data :: struct(),
  prepared()
) :: String.t()

DECLARE contents for a trigger function, or "" when the strategy needs no locals there. Sublimate.Projection omits the DECLARE block entirely when empty.

Examples

def declarations(:source, _projection_data, _prepared),
  do: "v_chunk_id integer;"

leaf_delete(projection_data, chain_spec, join, prepared)

@callback leaf_delete(
  projection_data :: struct(),
  chain_spec(),
  join :: join_spec(),
  prepared()
) :: branch_sql()

Returns the SQL that records the removal of a joined (leaf) row's contribution when that row is deleted.

This callback's SQL runs from a BEFORE DELETE trigger the engine installs on the leaf table - but only when the leaf's deletion is not already observable through the source triggers.

The engine decides this from the leaf table's inbound foreign key:

  • A reference that cascades (ON DELETE CASCADE) or nullifies (ON DELETE SET NULL) propagates the deletion back to the source, where the source delete/update callbacks already record it — so this callback is not wired up, avoiding a double-count.
  • A reference that does neither, or no foreign key, leaves the source's link dangling on delete, invisible to the source triggers - so this callback is wired up and is the sole recorder of the removal.

A strategy therefore does not decide whether leaf deletes are tracked; it only supplies the SQL for the case where they are. The chain gives the join path from source to leaf, and join is the leaf's own join, so the SQL can walk back from the deleted leaf row (OLD) to the affected source rows.

leaf_update(projection_data, chain_spec, join, prepared)

@callback leaf_update(
  projection_data :: struct(),
  chain_spec(),
  join :: join_spec(),
  prepared()
) :: branch_sql()

Body of a leaf table's AFTER UPDATE trigger function.

source_delete(projection_data, prepared)

@callback source_delete(projection_data :: struct(), prepared()) :: branch_sql()

Body of the source trigger's TG_OP = 'DELETE' branch.

source_insert(projection_data, prepared)

@callback source_insert(projection_data :: struct(), prepared()) :: branch_sql()

Body of the source trigger's TG_OP = 'INSERT' branch.

source_update(projection_data, prepared)

@callback source_update(projection_data :: struct(), prepared()) :: branch_sql()

Body of the source trigger's TG_OP = 'UPDATE' branch.

Types

branch_sql()

@type branch_sql() :: String.t()

SQL for one trigger branch. May contain several statements: a facet update emits a remove-old and an add-new. Sublimate.Projection splices it into the branch as-is, between THEN and the RETURN.

chain_spec()

@type chain_spec() :: %{field: term(), joins: [join_spec()], leaf_table: String.t()}

One field's linear chain: the field it serves, the joins from the source to the table its value is read from, and that leaf table.

A field whose value is on the source table itself has an empty joins list and its leaf_table is the source; it contributes no joined-table triggers.

join_spec()

@type join_spec() :: %{
  table: String.t(),
  match: String.t(),
  to: String.t(),
  where: String.t() | nil
}

join_triggers()

@type join_triggers() :: %{
  required(String.t()) => %{
    trigger: String.t(),
    trigger_fn: String.t(),
    delete_trigger: String.t()
  }
}

prepared()

@type prepared() :: term()

Opaque, strategy-specific data built by prepare/3.

trigger_kind()

@type trigger_kind() :: :source | :leaf | :link

Which trigger function's DECLARE block is being built.