Sublimate (Sublimate v0.1.4)

Copy Markdown

Sublimate is a helper library that builds database trigger logic for incremental updates to a destination table.

The data source can be a single source table or multiple joined tables. Changes to any of the source tables are staged to a deltas table. A merge function can then be applied - on demand or at scheduled intervals - to copy the changes to the destination table.

Sublimate maintains the destination table incrementally as the source changes; the consumer application/library defines what the destination contains and how staged changes merge into it.

One example of a consuming library is Refine ➚, which keeps a facets table up to date with changes in source tables.

Building blocks

Projection

The engine that builds and runs the infrastructure (deltas table, triggers) that keeps the destination table synchronized with its source tables, without knowing anything about the table's contents.

Projection-specific decisions are delegated to the projection's strategy (see below).

See: Sublimate.Projection

Strategy

The implementation of a specific projection. By implementing Sublimate.ProjectionStrategy callback functions, the consuming application/library instructs the projection how to keep the destination table synchronized.

See: Sublimate.ProjectionStrategy

Projection data

The attributes the projection operates on: the strategy module, and normalized and validated table and trigger references.

See: Sublimate.ProjectionData

Entry points

For a single projection, the lifecycle is:

install/2 is composed of two lower-level functions, exposed for advanced use:

Most consumers only need install/2. The two building blocks let a caller install several projections against one shared destination table - creating the shared infrastructure once, then installing each projection's triggers - which install/2, by recreating the deltas table and merge function on every call, cannot do.

See: Sublimate.Projection

Repo configuration

When calling Sublimate functions, the repo value in options can be omitted by passing otp_app in the projection_data parameter.

Sublimate reads the repo from the application's configuration under the :repo key:

config :my_app, repo: MyApp.Repo

Pass otp_app in the config given to ProjectionData.new/1, so Sublimate knows where to look:

config = %{
  source_table: "...",
  destination_table: "...",
  otp_app: :my_app,
  ...
}

If repo is given in the options, it is used directly; otherwise Sublimate falls back to the repo configured for otp_app.

Summary

Helpers

alias_column(qualified_column)

See Sublimate.SublimateHelpers.alias_column/1.

append_postgres_type(content, type)

See Sublimate.SublimateHelpers.append_postgres_type/2.

distinct_table_names?(qualified_table_name_a, qualified_table_name_b)

See Sublimate.SublimateHelpers.distinct_table_names?/2.

fetch_table_columns(table, repo, postgrex_options)

See Sublimate.SublimateHelpers.fetch_table_columns/3.

fetch_table_columns!(table, repo, postgrex_options)

See Sublimate.SublimateHelpers.fetch_table_columns!/3.

get_postgrex_options(database_options)

See Sublimate.SublimateHelpers.get_postgrex_options/1.

get_repo(opts)

See Sublimate.SublimateHelpers.get_repo/1.

identity_column_exists?(projection_data, table_columns)

See Sublimate.ProjectionHelpers.identity_column_exists?/2.

qualified_table_name(table_with_prefix)

See Sublimate.SublimateHelpers.qualified_table_name/1.

run_queries(queries, repo, postgrex_options)

See Sublimate.SublimateHelpers.run_queries/3.

run_query(sql, repo, postgrex_options, params \\ [])

See Sublimate.SublimateHelpers.run_query/4.

split_prefix_and_table_name(table_with_prefix)

See Sublimate.SublimateHelpers.split_prefix_and_table_name/1.

table_alias(qualified_table)

See Sublimate.SublimateHelpers.table_alias/1.

table_exists?(qualified_table_name, repo)

See Sublimate.SublimateHelpers.table_exists?/2.

validate_identifier(name)

See Sublimate.SublimateHelpers.validate_identifier/1.

wrap(content, wrap_chars)

See Sublimate.SublimateHelpers.wrap/2.

Projection data

create_projection_data(config)

See Sublimate.ProjectionData.new/1.

Projection helpers

maybe_add_identity_column(projection_data, table_columns, repo, postgrex_options)

See Sublimate.ProjectionHelpers.maybe_add_identity_column/4.

needs_identity_column?(projection_data, table_columns)

See Sublimate.ProjectionHelpers.needs_identity_column?/2.

Projection

changed_table_anchor(k_join, ref)

See Sublimate.Projection.changed_table_anchor/2.

column_name_of(qualified_column)

See Sublimate.Projection.column_name_of/1.

column_ref_table(qualified_column)

See Sublimate.Projection.column_ref_table/1.

create_shared_infrastructure(projection_data, prepared, merge_body, opts \\ [])

See Sublimate.Projection.create_shared_infrastructure/4.

emit_forward_joins(forward, link_join, ref)

See Sublimate.Projection.emit_forward_joins/3.

emit_joins(joins)

See Sublimate.Projection.emit_joins/1.

forward_joins_excluding_leaf(chain, leaf_table)

See Sublimate.Projection.forward_joins_excluding_leaf/2.

install(projection_data, opts \\ [])

See Sublimate.Projection.install/2.

install_triggers(projection_data, prepared, opts \\ [])

See Sublimate.Projection.install_triggers/3.

joins_sql_without_leaf(chain, qualified_source_table, leaf_join)

See Sublimate.Projection.joins_sql_without_leaf/3.

leaf_to_anchor(leaf_join, ref)

See Sublimate.Projection.leaf_to_anchor/2.

merge(projection_data, opts \\ [])

See Sublimate.Projection.merge/2.

same_table?(a, b)

See Sublimate.Projection.same_table?/2.

source_nearest_anchor(source_nearest, ref)

See Sublimate.Projection.source_nearest_anchor/2.

split_chain_at(chain, table)

See Sublimate.Projection.split_chain_at/2.

uninstall(projection_data, opts \\ [])

See Sublimate.Projection.uninstall/2.

uninstall_all(projection_datas, opts \\ [])

See Sublimate.Projection.uninstall_all/2.