erli18n_pt_store (erli18n v0.6.1)

Copy Markdown View Source

persistent_term storage layer for erli18n translation catalogs.

Each loaded {Domain, Locale} catalog is stored as ONE persistent term: key {erli18n_catalog, Domain, Locale}, value a single map holding

  • {singular, Context, Msgid} => Translation
  • {plural, Context, Msgid, Index} => Translation
  • '$header' => header_state()

Reads are copy-free (persistent_term:get/2 does not copy the term onto the caller's heap) and lock-free from the caller process. The read semantics mirror the previous ETS reads byte-for-byte: a missing catalog or a missing key both yield undefined, and the plural read evaluates the compiled rule (or the C/Germanic fallback) exactly as before.

Read-path rule (load-bearing)

Call persistent_term:get/2 FRESH on every lookup and let the returned map be transient. NEVER cache the catalog map in a long-lived process's state or process dictionary: a process holding the old term is forced into a major (fullsweep) garbage collection when the catalog is reloaded, and would serve a stale catalog. (erlang.org/doc/apps/erts/persistent_term.html, Best Practices.)

Reload cost

put_map/3 (install/reload) and unload/2 (persistent_term:erase/1) of the catalog map defer a node-wide literal-area cleanup: every process still referencing the old map runs a major GC, and all processes are made runnable to scan their heaps. It is paid once per (re)load — acceptable for erli18n's load-once-at-boot workload, but it is a real cost that ETS did not have.

The loaded-catalog index term (?INDEX_KEY) is written on the same node-wide basis only when its ordsets set actually changes: the first load of a {Domain, Locale} pair and the unload of the last reference to it each pay one index persistent_term:put (and its literal-area cleanup), but a reload of an already-indexed catalog skips the index write entirely (compare-before-put), so a steady-state reload pays the cost of the catalog-map term only.

Why the default-available index is NOT cached here

The per-request locale-negotiation default-available path rebuilds a canonical available_index from loaded_locales/0 on each request. Caching that prebuilt index as a SECOND persistent_term keyed off ?INDEX_KEY was considered and DELIBERATELY REJECTED. The per-request work it removes is marginal (one copy-free index_get/0 plus a usort/canonicalize over the tiny index) and is ALREADY fully avoidable by an application passing an explicit available option (the per-request thunk then returns it directly, with no rebuild). A cached term, by contrast, would add a recurring cost: a second persistent_term:put that must fire in lock-step with ?INDEX_KEY on EVERY index change (first load of a pair, last unload, and the header-only-reload index_del path), each scheduling the node-wide literal-area GC that the compare-before-put discipline exists to minimize. It would also widen the lock-step invariant surface (across put_map/3, unload/2, index_update/2, erase_all/0) and need ?INDEX_KEY-style namespace exclusion. A marginal, easily-avoided per-request saving traded for recurring reload-time node-wide GC plus added invariant/bug surface is premature optimization, so the index is recomputed per request rather than cached.

Summary

Functions

List every loaded catalog as {Domain, Locale, Map}. Filters the node's persistent terms by the erli18n_catalog namespace, so it is O(total persistent terms on the node) — an observability call, never the hot path.

Build the per-catalog value map from parsed .po entries and a header state, WITHOUT touching persistent_term (pure). Used to construct the map off the measured/serialized write path before a single put_map/3.

Number of data (non-header) keys in a catalog map.

The data (non-header) keys of a catalog map.

Erase every {erli18n_catalog, _, _} persistent term on the node and return how many were removed. Called on application stop, since persistent terms are node-global and are NOT cleared when the application stops.

Return the whole catalog map for {Domain, Locale}, or undefined if not loaded.

Plural-aware lookup. Mirrors erli18n_server:lookup_plural_form/5: read the header, select the form index via the compiled rule (or the C/Germanic fallback), then read the per-index plural key. Miss => undefined.

Total number of stored keys in a catalog map (data keys plus the header).

Build the catalog map and install it in one step (the install/load commit).

The loaded-locale set: the sorted, distinct locales across all loaded catalogs. ONE keyed persistent_term read (copy-free) plus a project/usort over the tiny index — O(1) on the node table, NOT the node-wide scan that all/0 does. This is the hot-path accessor behind erli18n:loaded_locales/0.

Header lookup. Mirrors erli18n_server:lookup_header/2 (miss => undefined).

Merge entries into the existing catalog map, creating it if absent and preserving the header if present. Used by the low-level insert API (erli18n_server:insert_singular/5 etc.): an empty merge is a no-op (no catalog is created, no put). A negative or non-integer plural form index is a loud function_clause crash, matching the historical insert contract.

Install a pre-built catalog map as the single persistent term for {Domain, Locale}.

Reload a catalog: whole-term replacement. Identical to load/4 because persistent_term:put/2 overwrites the term at the key, so there are no stale entries to prune (a concurrent reader sees either the entire old or the entire new catalog, never a half-applied one).

Approximate storage size of a catalog map in bytes (term word size * word).

Unload a catalog: erase the persistent term. Idempotent (ok whether present or not).

Types

catalog_map()

-type catalog_map() :: #{stored_key() => translation() | header_state()}.

context()

-type context() :: undefined | binary().

data_key()

-type data_key() :: {singular, context(), msgid()} | {plural, context(), msgid(), non_neg_integer()}.

domain()

-type domain() :: atom().

header_state()

-type header_state() :: erli18n_server:header_state().

locale()

-type locale() :: binary().

msgid()

-type msgid() :: binary().

stored_key()

-type stored_key() :: data_key() | '$header'.

translation()

-type translation() :: binary().

Functions

all()

-spec all() -> [{domain(), locale(), catalog_map()}].

List every loaded catalog as {Domain, Locale, Map}. Filters the node's persistent terms by the erli18n_catalog namespace, so it is O(total persistent terms on the node) — an observability call, never the hot path.

build_map(Entries, HeaderState)

-spec build_map([erli18n_po:entry()], header_state()) -> catalog_map().

Build the per-catalog value map from parsed .po entries and a header state, WITHOUT touching persistent_term (pure). Used to construct the map off the measured/serialized write path before a single put_map/3.

data_count(Map)

-spec data_count(catalog_map()) -> non_neg_integer().

Number of data (non-header) keys in a catalog map.

data_keys(Map)

-spec data_keys(catalog_map()) -> [data_key()].

The data (non-header) keys of a catalog map.

erase_all()

-spec erase_all() -> non_neg_integer().

Erase every {erli18n_catalog, _, _} persistent term on the node and return how many were removed. Called on application stop, since persistent terms are node-global and are NOT cleared when the application stops.

get_map(Domain, Locale)

-spec get_map(domain(), locale()) -> catalog_map() | undefined.

Return the whole catalog map for {Domain, Locale}, or undefined if not loaded.

get_plural_form(Domain, Locale, Context, Msgid, N)

-spec get_plural_form(domain(), locale(), context(), msgid(), integer()) ->
                         {ok, translation()} | undefined.

Plural-aware lookup. Mirrors erli18n_server:lookup_plural_form/5: read the header, select the form index via the compiled rule (or the C/Germanic fallback), then read the per-index plural key. Miss => undefined.

get_singular(Domain, Locale, Context, Msgid)

-spec get_singular(domain(), locale(), context(), msgid()) -> {ok, translation()} | undefined.

Singular lookup. Mirrors erli18n_server:lookup_singular/4 (miss => undefined).

key_count(Map)

-spec key_count(catalog_map()) -> non_neg_integer().

Total number of stored keys in a catalog map (data keys plus the header).

load(Domain, Locale, Entries, HeaderState)

-spec load(domain(), locale(), [erli18n_po:entry()], header_state()) -> ok.

Build the catalog map and install it in one step (the install/load commit).

loaded_locales()

-spec loaded_locales() -> [locale()].

The loaded-locale set: the sorted, distinct locales across all loaded catalogs. ONE keyed persistent_term read (copy-free) plus a project/usort over the tiny index — O(1) on the node table, NOT the node-wide scan that all/0 does. This is the hot-path accessor behind erli18n:loaded_locales/0.

lookup_header(Domain, Locale)

-spec lookup_header(domain(), locale()) -> {ok, header_state()} | undefined.

Header lookup. Mirrors erli18n_server:lookup_header/2 (miss => undefined).

merge_entries(Domain, Locale, Entries)

-spec merge_entries(domain(), locale(), [erli18n_po:entry()]) -> ok.

Merge entries into the existing catalog map, creating it if absent and preserving the header if present. Used by the low-level insert API (erli18n_server:insert_singular/5 etc.): an empty merge is a no-op (no catalog is created, no put). A negative or non-integer plural form index is a loud function_clause crash, matching the historical insert contract.

put_map(Domain, Locale, Map)

-spec put_map(domain(), locale(), catalog_map()) -> ok.

Install a pre-built catalog map as the single persistent term for {Domain, Locale}.

reload(Domain, Locale, Entries, HeaderState)

-spec reload(domain(), locale(), [erli18n_po:entry()], header_state()) -> ok.

Reload a catalog: whole-term replacement. Identical to load/4 because persistent_term:put/2 overwrites the term at the key, so there are no stale entries to prune (a concurrent reader sees either the entire old or the entire new catalog, never a half-applied one).

storage_bytes(Map)

-spec storage_bytes(catalog_map()) -> non_neg_integer().

Approximate storage size of a catalog map in bytes (term word size * word).

unload(Domain, Locale)

-spec unload(domain(), locale()) -> ok.

Unload a catalog: erase the persistent term. Idempotent (ok whether present or not).