Runlight.Store (runlight v0.3.2)

Copy Markdown View Source

Runlight's tables in a SQL database: SQLite, Postgres, or MySQL and MariaDB. The same tables, columns, and statements as the TypeScript SDK's SqlStore, so one database can be served by any implementation.

Make one over an app's Ecto repo with ecto/1:

Runlight.Store.ecto(repo: MyApp.Repo)

or over any Runlight.Db with new/2. Either takes prefix:, what every table and index name starts with in place of rl_, so two apps sharing one database each keep a set of their own: up to 32 letters, digits, and underscores, not starting with a digit.

Summary

Types

t()

A store. prefix is what its table and index names start with: rl_ unless it was given another. funnel_visits is the most visits a funnel is counted over; smaller in tests, to try the cap.

Functions

Writes a setting only if it is not there yet. Of two callers at once, only one gets true.

The top values of a dimension, limit from offset.

Adds up one local day of a site: totals, each visit dimension, and pages. A visit belongs to the day it started.

Records a period as sent. Only one caller wins, so two cron runs at once cannot both send it.

Forgets built days, all of a site's (no options), those starting before before:, or those touching from: to to:, so they are built again.

What every table and index name starts with, unless a store is given another prefix in its place.

Hides a link and frees its slug; its clicks stay in the history.

Deleting a share is how it is revoked: the link stops working at once.

Deletes a site and everything recorded for it. Its events and visits go a day at a time first, and what is left goes in one transaction.

Deleting a token is how it is revoked: it stops working at once.

Deletes a site's visits and events from before a time, for its retention setting.

Deletes a site's events from from on whose visit no longer exists, a day at a time.

Deletes every salt but those of these days, so the hashes made with them can never be made again.

A store over an app's Ecto repo; see Runlight.Db.Ecto for the options, and new/2 for prefix:.

The property names sent with an event in a query's range, most used first.

The values one property of an event took, with how often and by how many visitors.

When Runlight itself first counted a visit, leaving out imported history.

When the site's first visit was recorded, or nil with no data yet.

How many visits reached each step, in order, within the same visit. Step one is the first matching row in the range; each later step must come after the step before it. Filters choose which visits enter the funnel. Past funnel_visits visits in the range, only the newest are counted, and sampled says so.

A goal's conversions split by where the visit came from, or by the page it happened on.

A goal's conversions and revenue in each bucket, by when each visit started.

One goal's conversions, converting visitors, and revenue for a query's range and filters.

Every goal's totals in one pass over the range's events, by goal id.

Seeks one matching event per hostname, skipping its remaining events before the next seek.

Visits by quarter hour since the epoch, which the caller folds into local weekdays and hours, keeping time zones out of SQL.

Records an event; row is a map with the EventRow's fields as atoms in snake case.

Records a new session; row is a map with the SessionRow's fields as atoms in snake case.

Each visit's pageviews in order, at most per_visit of them, for journeys. Visits belong to the range they started in.

When the site last recorded a visit, or nil if it never has.

One link's clicks by a visit dimension.

The live link with a slug. Slugs are unique across every domain.

One link's clicks per bucket.

A site's links, newest first, with their clicks in a range.

How long the stores wait for the lock for creating tables, in milliseconds: 30 seconds, unless a test has set another with lock_wait_in_tests/1.

Creates the tables, and brings an older database up to date. Safe to call any number of times.

A store over a Runlight.Db. prefix: takes the place of rl_ at the start of every table and index name, so two apps can each keep Runlight's tables in one database. A Db that is already prefixed keeps its own.

The visitor's open session: any of their hashes, active since since.

Keeps SQLite's planner statistics current, which it never gathers by itself. A sample of each index is enough. Postgres gathers its own.

The pageview an engagement ping or event belongs to, with when its visit started and was last active.

What the store's table and index names start with: rl_ unless it was given another.

Who is on the site now, what they read, and the latest activity.

Puts a period back when its email failed, so the next run tries again.

Removes a domain. Its links keep it as their home and fall back to the app's own link path until it is added again.

The days of a site already built.

The salt for a day, made on first ask. Two racing callers agree on one.

The headline numbers for each bucket.

A setting that is made once: every caller gets the value written first, whoever made it.

Every setting whose key starts with a prefix, such as each connected install's.

After opening a visit: the visitor's earliest visit still going, which is this one unless another process opened one for the same visitor at the same moment. Then both settle on the earliest, and the one that is not drops its own, before anything is counted into it.

Settings changed in the dashboard, by site. They win over the ones in code.

The sites kept in the database, by name.

The headline numbers for a query.

Changes a setting only if it still holds from. Of two callers at once, only one gets true.

Runlight's tables, after the prefix.

The store's tables as named in the database, for an app removing Runlight: each one by name, since another app's tables may well start with the same letters.

Reads a setting and deletes it. Of two callers at once, only the one whose delete took the row gets its value.

Counts a row into its session. An event with reopen false, one that joins a visit already ended, counts without moving the session's last activity.

After a late event or engagement ping joins an old visit, the day that visit started may already be added up. Forget that day so the next check builds it again.

Runs fun with a store whose every query is in one transaction.

Whether a visit to one of these sites that started in a stretch of time was still going at since.

Just the visitor count from stats, in one query, for conversion rates.

Types

t()

@type t() :: %Runlight.Store{
  db: Runlight.Db.t(),
  funnel_visits: pos_integer(),
  prefix: String.t()
}

A store. prefix is what its table and index names start with: rl_ unless it was given another. funnel_visits is the most visits a funnel is counted over; smaller in tests, to try the cap.

Functions

add_engagement(store, id, ms)

@spec add_engagement(t(), String.t(), integer()) :: :ok

add_setting(store, key, value)

@spec add_setting(t(), String.t(), String.t()) :: boolean()

Writes a setting only if it is not there yet. Of two callers at once, only one gets true.

breakdown(store, query, dimension, limit, offset)

@spec breakdown(t(), map(), String.t(), pos_integer(), non_neg_integer()) :: [
  Runlight.JS.Object.t()
]

The top values of a dimension, limit from offset.

build_rollup_day(store, site, day, start, finish)

@spec build_rollup_day(t(), String.t(), String.t(), integer(), integer()) :: :ok

Adds up one local day of a site: totals, each visit dimension, and pages. A visit belongs to the day it started.

claim_report(store, id, period, now)

@spec claim_report(t(), String.t(), String.t(), integer()) :: boolean()

Records a period as sent. Only one caller wins, so two cron runs at once cannot both send it.

clear_rollups(store, site, range \\ [])

@spec clear_rollups(t(), String.t(), keyword()) :: :ok

Forgets built days, all of a site's (no options), those starting before before:, or those touching from: to to:, so they are built again.

default_prefix()

@spec default_prefix() :: String.t()

What every table and index name starts with, unless a store is given another prefix in its place.

delete_funnel(store, id)

@spec delete_funnel(t(), String.t()) :: :ok

delete_goal(store, id)

@spec delete_goal(t(), String.t()) :: :ok

delete_link(store, id, now)

@spec delete_link(t(), String.t(), integer()) :: :ok

Hides a link and frees its slug; its clicks stay in the history.

delete_report(store, id)

@spec delete_report(t(), String.t()) :: :ok

delete_share(store, id)

@spec delete_share(t(), String.t()) :: :ok

Deleting a share is how it is revoked: the link stops working at once.

delete_site(store, id)

@spec delete_site(t(), String.t()) :: :ok

Deletes a site and everything recorded for it. Its events and visits go a day at a time first, and what is left goes in one transaction.

delete_token(store, id)

@spec delete_token(t(), String.t()) :: boolean()

Deleting a token is how it is revoked: it stops working at once.

drop_before(store, site, ts)

@spec drop_before(t(), String.t(), integer()) :: :ok

Deletes a site's visits and events from before a time, for its retention setting.

drop_orphans(store, site, from, until)

@spec drop_orphans(t(), String.t(), integer(), integer()) :: :ok

Deletes a site's events from from on whose visit no longer exists, a day at a time.

drop_salts_except(store, days)

@spec drop_salts_except(t(), [String.t()]) :: :ok

Deletes every salt but those of these days, so the hashes made with them can never be made again.

ecto(opts)

@spec ecto(keyword()) :: t()

A store over an app's Ecto repo; see Runlight.Db.Ecto for the options, and new/2 for prefix:.

event_prop_keys(store, query, event)

@spec event_prop_keys(t(), map(), String.t()) :: [Runlight.JS.Object.t()]

The property names sent with an event in a query's range, most used first.

event_prop_values(store, query, event, key, limit)

@spec event_prop_values(t(), map(), String.t(), String.t(), pos_integer()) :: [
  Runlight.JS.Object.t()
]

The values one property of an event took, with how often and by how many visitors.

first_own_visit(store, site)

@spec first_own_visit(t(), String.t()) :: number() | nil

When Runlight itself first counted a visit, leaving out imported history.

first_seen(store, site)

@spec first_seen(t(), String.t()) :: number() | nil

When the site's first visit was recorded, or nil with no data yet.

funnel_counts(store, query, funnel)

@spec funnel_counts(t(), map(), Runlight.JS.Object.t()) :: %{
  counts: [non_neg_integer()],
  sampled: boolean()
}

How many visits reached each step, in order, within the same visit. Step one is the first matching row in the range; each later step must come after the step before it. Filters choose which visits enter the funnel. Past funnel_visits visits in the range, only the newest are counted, and sampled says so.

funnels(store, site)

@spec funnels(t(), String.t()) :: [Runlight.JS.Object.t()]

goal_breakdown(store, query, goal, by, limit \\ 10)

@spec goal_breakdown(t(), map(), Runlight.JS.Object.t(), String.t(), pos_integer()) ::
  [
    Runlight.JS.Object.t()
  ]

A goal's conversions split by where the visit came from, or by the page it happened on.

goal_by_id(store, id)

@spec goal_by_id(t(), String.t()) :: Runlight.JS.Object.t() | nil

goal_series(store, query, goal, buckets)

@spec goal_series(t(), map(), Runlight.JS.Object.t(), [map()]) :: [
  Runlight.JS.Object.t()
]

A goal's conversions and revenue in each bucket, by when each visit started.

goal_totals(store, query, goal)

@spec goal_totals(t(), map(), Runlight.JS.Object.t()) :: Runlight.JS.Object.t()

One goal's conversions, converting visitors, and revenue for a query's range and filters.

goal_totals_all(store, query, goals)

@spec goal_totals_all(t(), map(), [Runlight.JS.Object.t()]) :: Runlight.JS.Object.t()

Every goal's totals in one pass over the range's events, by goal id.

goals(store, site \\ nil)

@spec goals(t(), String.t() | nil) :: [Runlight.JS.Object.t()]

hostname_suggestions(store, query, limit)

@spec hostname_suggestions(t(), map(), pos_integer()) :: [Runlight.JS.Object.t()]

Seeks one matching event per hostname, skipping its remaining events before the next seek.

hourly(store, query)

@spec hourly(t(), map()) :: [map()]

Visits by quarter hour since the epoch, which the caller folds into local weekdays and hours, keeping time zones out of SQL.

insert_event(store, row)

@spec insert_event(t(), map()) :: :ok

Records an event; row is a map with the EventRow's fields as atoms in snake case.

insert_link(store, l)

@spec insert_link(t(), Runlight.JS.Object.t()) :: :ok

insert_report(store, r)

@spec insert_report(t(), Runlight.JS.Object.t()) :: :ok

insert_session(store, row)

@spec insert_session(t(), map()) :: :ok

Records a new session; row is a map with the SessionRow's fields as atoms in snake case.

insert_share(store, s)

@spec insert_share(t(), Runlight.JS.Object.t()) :: :ok

insert_token(store, t)

@spec insert_token(t(), Runlight.JS.Object.t()) :: :ok

journey_pages(store, query, per_visit)

@spec journey_pages(t(), map(), pos_integer()) :: %{rows: [map()], sampled: boolean()}

Each visit's pageviews in order, at most per_visit of them, for journeys. Visits belong to the range they started in.

last_seen(store, site)

@spec last_seen(t(), String.t()) :: number() | nil

When the site last recorded a visit, or nil if it never has.

links(store, site, from, to)

@spec links(t(), String.t(), integer(), integer()) :: [Runlight.JS.Object.t()]

A site's links, newest first, with their clicks in a range.

lock_wait_ms()

@spec lock_wait_ms() :: pos_integer()

How long the stores wait for the lock for creating tables, in milliseconds: 30 seconds, unless a test has set another with lock_wait_in_tests/1.

migrate(store)

@spec migrate(t()) :: :ok

Creates the tables, and brings an older database up to date. Safe to call any number of times.

new(db, opts \\ [])

@spec new(Runlight.Db.t(), keyword()) :: t()

A store over a Runlight.Db. prefix: takes the place of rl_ at the start of every table and index name, so two apps can each keep Runlight's tables in one database. A Db that is already prefixed keeps its own.

open_session(store, site, visitors, since)

@spec open_session(t(), String.t(), [String.t()], integer()) :: map() | nil

The visitor's open session: any of their hashes, active since since.

optimize(store, only_when_stale \\ false)

@spec optimize(t(), boolean()) :: :ok

Keeps SQLite's planner statistics current, which it never gathers by itself. A sample of each index is enough. Postgres gathers its own.

With only_when_stale, only when there are none, or the events have more than doubled since they were gathered. Statistics gathered on a new, empty database say every table is empty, and a goal over a million visits then took a minute rather than milliseconds.

pageview(store, site, pageview)

@spec pageview(t(), String.t(), String.t()) :: map() | nil

The pageview an engagement ping or event belongs to, with when its visit started and was last active.

prefix(store)

@spec prefix(t()) :: String.t()

What the store's table and index names start with: rl_ unless it was given another.

realtime(store, site, now)

@spec realtime(t(), String.t(), integer()) :: Runlight.JS.Object.t()

Who is on the site now, what they read, and the latest activity.

release_report(store, id, period, previous)

@spec release_report(t(), String.t(), String.t(), String.t()) :: :ok

Puts a period back when its email failed, so the next run tries again.

rename_share(store, id, name)

@spec rename_share(t(), String.t(), String.t()) :: :ok

report_by(store, field, value)

@spec report_by(t(), :id | :token, String.t()) :: Runlight.JS.Object.t() | nil

reports(store, site \\ nil)

@spec reports(t(), String.t() | nil) :: [Runlight.JS.Object.t()]

rollup_days(store, site)

@spec rollup_days(t(), String.t()) :: MapSet.t()

The days of a site already built.

salt(store, day, fresh)

@spec salt(t(), String.t(), String.t()) :: String.t()

The salt for a day, made on first ask. Two racing callers agree on one.

salt_if_exists(store, day)

@spec salt_if_exists(t(), String.t()) :: String.t() | nil

save_funnel(store, f)

@spec save_funnel(t(), Runlight.JS.Object.t()) :: :ok

save_goal(store, g, before \\ nil)

@spec save_goal(t(), Runlight.JS.Object.t(), Runlight.JS.Object.t() | nil) :: :ok

series(store, query, buckets)

@spec series(t(), map(), [map()]) :: [Runlight.JS.Object.t()]

The headline numbers for each bucket.

set_setting(store, key, value)

@spec set_setting(t(), String.t(), String.t() | nil) :: :ok

set_site_overrides(store, id, overrides)

@spec set_site_overrides(t(), String.t(), Runlight.JS.Object.t()) :: :ok

setting(store, key)

@spec setting(t(), String.t()) :: String.t() | nil

setting_or_add(store, key, make)

@spec setting_or_add(t(), String.t(), (-> String.t())) :: String.t()

A setting that is made once: every caller gets the value written first, whoever made it.

settings_starting_with(store, prefix)

@spec settings_starting_with(t(), String.t()) :: [
  %{key: String.t(), value: String.t()}
]

Every setting whose key starts with a prefix, such as each connected install's.

settle_open_session(store, site, visitors, since, opened)

@spec settle_open_session(t(), String.t(), [String.t()], integer(), map()) :: map()

After opening a visit: the visitor's earliest visit still going, which is this one unless another process opened one for the same visitor at the same moment. Then both settle on the earliest, and the one that is not drops its own, before anything is counted into it.

share_by_id(store, id)

@spec share_by_id(t(), String.t()) :: Runlight.JS.Object.t() | nil

shares(store, site)

@spec shares(t(), String.t()) :: [Runlight.JS.Object.t()]

site_overrides(store)

@spec site_overrides(t()) :: %{required(String.t()) => Runlight.JS.Object.t()}

Settings changed in the dashboard, by site. They win over the ones in code.

sites(store)

@spec sites(t()) :: [Runlight.JS.Object.t()]

The sites kept in the database, by name.

stats(store, query)

@spec stats(t(), map()) :: Runlight.JS.Object.t()

The headline numbers for a query.

swap_setting(store, key, from, to)

@spec swap_setting(t(), String.t(), String.t(), String.t()) :: boolean()

Changes a setting only if it still holds from. Of two callers at once, only one gets true.

table_names()

@spec table_names() :: [String.t()]

Runlight's tables, after the prefix.

tables(store)

@spec tables(t()) :: [String.t()]

The store's tables as named in the database, for an app removing Runlight: each one by name, since another app's tables may well start with the same letters.

take_setting(store, key)

@spec take_setting(t(), String.t()) :: String.t() | nil

Reads a setting and deletes it. Of two callers at once, only the one whose delete took the row gets its value.

token_by_hash(store, hash)

@spec token_by_hash(t(), String.t()) :: Runlight.JS.Object.t() | nil

tokens(store)

@spec tokens(t()) :: [Runlight.JS.Object.t()]

touch_session(store, id, ts, kind, path, reopen \\ true)

@spec touch_session(t(), String.t(), integer(), String.t(), String.t(), boolean()) ::
  :ok

Counts a row into its session. An event with reopen false, one that joins a visit already ended, counts without moving the session's last activity.

touch_token(store, id, now)

@spec touch_token(t(), String.t(), integer()) :: :ok

touched_old_visit(store, site, started, before)

@spec touched_old_visit(t(), String.t(), integer(), integer()) :: :ok

After a late event or engagement ping joins an old visit, the day that visit started may already be added up. Forget that day so the next check builds it again.

transaction(store, fun)

@spec transaction(t(), (t() -> term())) :: term()

Runs fun with a store whose every query is in one transaction.

update_link(store, l)

@spec update_link(t(), Runlight.JS.Object.t()) :: :ok

upsert_site(store, site, now)

@spec upsert_site(t(), Runlight.JS.Object.t(), integer()) :: :ok

visit_open?(store, sites, from, to, since)

@spec visit_open?(t(), [String.t()], integer(), integer(), integer()) :: boolean()

Whether a visit to one of these sites that started in a stretch of time was still going at since.

visitors(store, query)

@spec visitors(t(), map()) :: number()

Just the visitor count from stats, in one query, for conversion rates.