erli18n_server (erli18n v0.1.0)

Copy Markdown View Source

Catalog gen_server: owner and sole writer of the translations ETS table.

What it is and which problem it solves

This module is the heart of the erli18n runtime: it loads .po catalogs, keeps the translations live in ETS and answers lookups (singular, plural and header). The central problem it solves is reconciling two contradictory requirements: translation reads are extremely hot (every UI string goes through a lookup) and must be lock-free; writes, on the other hand, must be serialized to keep the protected ETS consistent. The solution is a strict split between the write path (serialized by this process's mailbox) and the read path (straight from ETS, with no roundtrip to the server).

Mental model

Think of this module in three layers:

  1. Read path (hot path, lock-free). lookup_singular/4, lookup_plural_form/5 and lookup_header/2 run ets:lookup/2 directly in the CALLING process — no message reaches the server mailbox. That is why N processes can read in parallel with no bottleneck (RISK-012). The ETS is protected: any process reads, only the owner (this server) writes.

  2. Write path (serialized). insert_*, unload/2 and the load commits become gen_server:call/2,3; handle_call/3 is the only critical section where the ETS is mutated. Since ETS-set performs atomic per-row inserts under the single mailbox, there is never observable mixed state.

  3. Load orchestration (heavy work OUTSIDE the mailbox). ensure_loaded/4 and reload/4 run the heavy, failable phase (size-check, read, parse, compile of the plural rule, CLDR divergence, object build, fuzzy count) in the CALLING process, producing a pure in-memory staged(). Only this validated payload travels to the server for a millisecond-scale commit. This way a large/slow/pathological .po from one tenant never blocks another's load (finding #6).

Trusted vs untrusted. A .po's Plural-Forms rule is untrusted input; that is why it is compiled with bounds (see erli18n_plural) and lookup_plural_form/5 wraps the evaluation in a belt-and-suspenders try. This module's anti-DoS bounds (max_bytes, max_entries) reject large catalogs BEFORE any ETS mutation.

ETS-TRANSFER / heir. This server does NOT create the data table. A dedicated owner (erli18n_table_owner, started before it under rest_for_one) creates the table and hands it over via give_away/3 ('ETS-TRANSFER' consumed in init/1), remaining as heir. If this worker crashes, the table returns INTACT to the owner instead of being destroyed — every loaded catalog survives the restart. On reclaiming it, init/1 rebuilds the O(1) catalog index from the surviving rows.

Two mental maps of "state". This gen_server's State is an empty map #{} — that is deliberate. The truth lives in TWO ETS tables: the data table (?ETS_TABLE, inherited from the owner) and a server-owned secondary index (?CATALOG_INDEX_TABLE) that stores, per catalog, the set of data keys. The index makes memory_info/0 O(1) and unload/2 O(catalog size) instead of O(total rows) scans (findings #7/#13).

When and how a dev touches this module

Quickstart

1> application:ensure_all_started(erli18n).
{ok, [erli18n]}
2> Po = erli18n_server:default_po_path(my_app, my_domain, <<"fr">>).
"/.../priv/locale/fr/LC_MESSAGES/my_domain.po"
3> erli18n_server:ensure_loaded(my_domain, <<"fr">>, Po).
{ok, 128}
4> erli18n_server:ensure_loaded(my_domain, <<"fr">>, Po).
{ok, already}
5> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Hello">>).
{ok, <<"Bonjour">>}
6> erli18n_server:lookup_plural_form(my_domain, <<"fr">>, undefined, <<"file">>, 2).
{ok, <<"fichiers">>}
7> erli18n_server:memory_info().
#{ets_bytes => 24576, num_catalogs => 1, num_keys => 131}

(num_keys counts ALL rows, including the header; loaded_catalogs/0 counts only the 130 data rows — see both functions.)

Main entry points

Summary

Types

Errors from the anti-DoS bounds (finding #6), a subset of ensure_error(). Both are surfaced in the CALLER's heavy phase, BEFORE any ETS mutation: input_too_large when the file size exceeds max_bytes (checked without reading the bytes, via filelib:file_size/1); too_many_entries when the post-parse count exceeds max_entries. The second element is the observed value, the third is the configured limit.

Union of all structured errors a load can return. Each variant maps a failable step of the stage pipeline (in the order it can fail): an I/O error reading the file ({file_error, _}), a .po parse error (erli18n_po:parse_error()), a plural-rule compile error ({plural_compile_error, _}) and the anti-DoS caps (bound_error()). None of them leaves the ETS mutated.

Result of a load (ensure_loaded/3,4, reload/3,4).

Header state of a loaded catalog, returned by lookup_header/2 and stored in the ?HEADER_KEY row. The PRESENCE of this row is the idempotency signal used by ensure_loaded/3 ("catalog already loaded").

A catalog to load in the bulk API ensure_loaded_many/1. Same positional shape as the ensure_loaded/4 arguments: {Domain, Locale, PoPath, Opts}.

Load options accepted by ensure_loaded/4, reload/4 and each item of ensure_loaded_many/1. All fields are optional; omitting one preserves the legacy behaviour (modulo the safety-cap defaults).

Functions

No state migration: the State is an empty #{} (the truth lives in ETS), so a code upgrade has no state to transform. Returns {ok, State}.

Computes the conventional gettext .po path for an application.

Idempotent load of a .po catalog. Same as ensure_loaded/4 with #{}.

Idempotent load of a .po catalog with resource options.

Bulk load of N catalogs with a single commit on the server.

Serialized critical section — NOTE FOR THE MAINTAINER. This is the ONLY place where the ETS is mutated; every write in the module passes through here under the single mailbox, so there is never observable mixed state (ETS-set inserts atomically per row).

Inert by design: this server uses no casts (every write is a synchronous call, so the caller gets acknowledgement and backpressure). It exists only to satisfy the gen_server behaviour. Messages are ignored with {noreply, State}.

Inert by design: this server expects no out-of-band messages. The initial 'ETS-TRANSFER' is consumed in init/1 by an explicit receive, not here; a return 'ETS-TRANSFER' after a crash reaches the OWNER (heir), not this worker. Messages are ignored with {noreply, State}.

Initialization callback — NOTE FOR THE MAINTAINER (do not call by hand; it is the supervisor that invokes it via start_link/0).

Inserts a batch of entries (singular and plural) of a catalog in one write.

Inserts/overwrites the plural forms of a Msgid, serialized by the server.

Inserts/overwrites a singular translation, serialized by the server.

Lists the loaded catalogs with the data-row count of each.

Lock-free lookup of the (Domain, Locale) catalog header, straight from ETS.

The CORRECT entry point for plural reads (form-aware, lock-free).

Lock-free lookup of a singular translation, straight from ETS in the caller.

Returns the memory usage of the translations ETS table.

Atomic reload of a .po catalog. Same as reload/4 with #{}.

Atomic reload of a .po catalog with resource options (STAGE -> SWAP).

Starts the catalog gen_server, registered locally as erli18n_server.

No cleanup to do: the data table is protected and has a heir (the owner), so on a crash of this worker it returns INTACT to the owner instead of being destroyed — exactly the scenario this architecture protects. The secondary index is server-owned and dies with the worker, being rebuilt on the next init/1. Returns ok.

Removes the (Domain, Locale) catalog entirely (entries + header).

Enumerates the keys (singular and plural) loaded for (Domain, Locale).

Types

bound_error()

-type bound_error() ::
          {input_too_large, Bytes :: non_neg_integer(), Limit :: non_neg_integer()} |
          {too_many_entries, Count :: non_neg_integer(), Limit :: non_neg_integer()}.

Errors from the anti-DoS bounds (finding #6), a subset of ensure_error(). Both are surfaced in the CALLER's heavy phase, BEFORE any ETS mutation: input_too_large when the file size exceeds max_bytes (checked without reading the bytes, via filelib:file_size/1); too_many_entries when the post-parse count exceeds max_entries. The second element is the observed value, the third is the configured limit.

catalog_entry()

-type catalog_entry() :: singular_entry() | plural_entry().

context()

-type context() :: undefined | binary().

divergence_info()

-type divergence_info() :: none | {plural_divergence, binary(), binary()}.

domain()

-type domain() :: atom().

ensure_error()

-type ensure_error() ::
          erli18n_po:parse_error() |
          {plural_compile_error, erli18n_plural:compile_error()} |
          {file_error, file:posix() | badarg | terminated | system_limit} |
          bound_error() |
          {load_failed, term()}.

Union of all structured errors a load can return. Each variant maps a failable step of the stage pipeline (in the order it can fail): an I/O error reading the file ({file_error, _}), a .po parse error (erli18n_po:parse_error()), a plural-rule compile error ({plural_compile_error, _}) and the anti-DoS caps (bound_error()). None of them leaves the ETS mutated.

ensure_result()

-type ensure_result() ::
          {ok, NewlyLoaded :: non_neg_integer()} | {ok, already} | {error, ensure_error()}.

Result of a load (ensure_loaded/3,4, reload/3,4).

  • {ok, NewlyLoaded}: a real load — number of entries parsed, compiled and installed.
  • {ok, already}: idempotent fast-path, the catalog was already loaded (only ensure_loaded/ensure_loaded_many; reload never returns this).
  • {error, ensure_error()}: structured error; the ETS stays intact (all errors occur BEFORE any mutation).

header_state()

-type header_state() ::
          #{plural := erli18n_plural:plural_compiled() | fallback,
            plural_raw := binary(),
            po_path := file:filename(),
            loaded_at := integer(),
            divergence := divergence_info(),
            fuzzy_included := boolean(),
            num_entries := non_neg_integer()}.

Header state of a loaded catalog, returned by lookup_header/2 and stored in the ?HEADER_KEY row. The PRESENCE of this row is the idempotency signal used by ensure_loaded/3 ("catalog already loaded").

  • plural: the ALREADY compiled Plural-Forms rule (erli18n_plural:plural_compiled()), or the atom fallback when the .po came without a plural header (the lookup then uses the C/Germanic default, see lookup_plural_form/5).
  • plural_raw: the raw text of the rule (or the fallback rule) for observability/round-trip.
  • po_path: the path of the source .po.
  • loaded_at: erlang:system_time(millisecond) at the moment of the load.
  • divergence: divergence_info()none or the vs-CLDR divergence warning.
  • fuzzy_included: whether the load included #, fuzzy entries.
  • num_entries: entry count (singular + plural aggregated as the parser counts them), the number reported in {ok, NewlyLoaded}.

load_spec()

-type load_spec() :: {domain(), locale(), file:filename(), opts()}.

A catalog to load in the bulk API ensure_loaded_many/1. Same positional shape as the ensure_loaded/4 arguments: {Domain, Locale, PoPath, Opts}.

locale()

-type locale() :: binary().

msgid()

-type msgid() :: binary().

msgid_plural()

-type msgid_plural() :: undefined | binary().

opts()

-type opts() ::
          #{include_fuzzy => boolean(),
            max_bytes => non_neg_integer() | infinity,
            max_entries => non_neg_integer() | infinity,
            timeout => timeout()}.

Load options accepted by ensure_loaded/4, reload/4 and each item of ensure_loaded_many/1. All fields are optional; omitting one preserves the legacy behaviour (modulo the safety-cap defaults).

  • include_fuzzy (default false): includes entries marked #, fuzzy.
  • max_bytes (default application:get_env(erli18n, max_po_bytes), 16 MiB): rejects the file BEFORE reading it whole (via filelib:file_size/1). infinity disables the cap.
  • max_entries (default application:get_env(erli18n, max_po_entries), 500000): rejects the catalog AFTER the parse if it has more than N entries. infinity disables the cap.
  • timeout (default 5000 ms): deadline of the gen_server:call/3 that performs the commit. Since the heavy phase no longer runs behind the mailbox, the deadline covers only the bulk insert (millisecond scale).

plural_entries()

-type plural_entries() :: [{plural_index(), translation()}].

plural_entry()

-type plural_entry() :: {plural, context(), msgid(), msgid_plural(), plural_entries()}.

plural_index()

-type plural_index() :: non_neg_integer().

singular_entry()

-type singular_entry() :: {singular, context(), msgid(), translation()}.

translation()

-type translation() :: binary().

Functions

code_change(OldVsn, State, Extra)

No state migration: the State is an empty #{} (the truth lives in ETS), so a code upgrade has no state to transform. Returns {ok, State}.

default_po_path(App, Domain, Locale)

-spec default_po_path(atom(), domain(), locale()) -> file:filename().

Computes the conventional gettext .po path for an application.

Parameters

  • App: the OTP application whose priv contains the catalogs (resolved via code:priv_dir/1).
  • Domain: the gettext domain (becomes the file name <Domain>.po).
  • Locale: the binary locale (becomes the directory segment <Locale>).

Return

Returns <priv>/locale/<Locale>/LC_MESSAGES/<Domain>.po (a string). This function only COMPOSES the path — it does not check whether the file exists.

Failure modes

Crashes with {priv_dir_not_found, App} if the application is unknown (code:priv_dir/1 returns {error, bad_name}) — so the operator sees the misconfiguration immediately, instead of loading a path with {error, bad_name} embedded. Separation of concerns: ensure_loaded/3 takes the path explicitly; there is no implicit resolution here — it is the façade that decides whether to honour this convention or use a caller-supplied path.

1> erli18n_server:default_po_path(my_app, my_domain, <<"fr">>).
"/path/to/my_app/priv/locale/fr/LC_MESSAGES/my_domain.po"

See ensure_loaded/3.

ensure_loaded(Domain, Locale, PoPath)

-spec ensure_loaded(domain(), locale(), file:filename()) -> ensure_result().

Idempotent load of a .po catalog. Same as ensure_loaded/4 with #{}.

If (Domain, Locale) is already loaded (header present), returns {ok, already} without touching disk. Otherwise it runs the full pipeline (read file, parse, compile the plural rule, validate against CLDR as a WARNING — divergence never blocks the load, install in ETS atomically) and returns {ok, NewlyLoaded} with the number of inserted entries, or {error, ensure_error()} leaving the ETS INTACT.

1> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "priv/locale/fr/LC_MESSAGES/my_domain.po").
{ok, 128}
2> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "priv/locale/fr/LC_MESSAGES/my_domain.po").
{ok, already}
3> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "/no/such/file.po").
{error, {file_error, enoent}}

See ensure_loaded/4 (options and bounds), reload/3 (forces reinstall), ensure_loaded_many/1 (batch).

ensure_loaded(Domain, Locale, PoPath, Opts)

-spec ensure_loaded(domain(), locale(), file:filename(), opts()) -> ensure_result().

Idempotent load of a .po catalog with resource options.

Idempotent fast-path: if the catalog is already loaded, returns {ok, already} via a pure ETS read (no disk, no server roundtrip). On a miss, the heavy phase (read+parse+compile+validate+bounds) runs in the CALLING process, inside the span [erli18n, catalog, load], and only the validated payload is handed to the server for the millisecond commit.

Opts (all optional):

  • include_fuzzy (default false): includes entries marked #, fuzzy.
  • max_bytes (non_neg_integer() | infinity): rejects the file BEFORE reading it whole (via filelib:file_size/1) if it exceeds the limit; default comes from application:get_env(erli18n, max_po_bytes) (16 MiB). infinity = no cap.

  • max_entries (non_neg_integer() | infinity): rejects the catalog AFTER the parse if it has more than N entries; default application:get_env(erli18n, max_po_entries) (500000). infinity = no cap.

  • timeout (timeout()): deadline of the commit gen_server:call/3 (default 5000 ms; the commit is only the bulk insert, ~26 ms for 40k entries).

Returns {ok, NewlyLoaded}, {ok, already} or {error, ensure_error()} (including {input_too_large, _, _} / {too_many_entries, _, _}), always before any ETS mutation.

Edge cases

  • Check-then-insert race: the idempotent fast-path reads outside the serialization, but the commit RE-CHECKS idempotency under the mailbox (mode ensure), so two concurrent callers of the same catalog do not overwrite each other — the second sees {ok, already}.
  • CLDR divergence: never an error; emits a warning log/telemetry and proceeds, storing the divergence in the header_state().
  • timeout exceeded: the commit gen_server:call/3 may crash with {timeout, _}; since the commit is only the bulk insert, this practically only happens with a very tight timeout.
1> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "fr.po", #{include_fuzzy => true}).
{ok, 131}
2> erli18n_server:ensure_loaded(my_domain, <<"de">>, "big.po", #{max_bytes => 1024}).
{error, {input_too_large, 6553600, 1024}}

See ensure_loaded/3, reload/4, ensure_loaded_many/1, opts(), ensure_error().

ensure_loaded_many(Specs)

-spec ensure_loaded_many([load_spec()]) -> [{domain(), locale(), ensure_result()}].

Bulk load of N catalogs with a single commit on the server.

Specs is [{Domain, Locale, PoPath, Opts}]. The heavy phase of each spec runs in the calling process (sequential preparation — the v0.1 trade-off; a parallel fan-out is a future evolution) and all ready payloads are delivered in a SINGLE commit, collapsing N roundtrips into one and making the per-catalog idempotency check O(1) (finding #7's O(1) index). Each Opts follows ensure_loaded/4.

Returns [{Domain, Locale, ensure_result()}] — already-loaded or failed catalogs are reported individually; one catalog's error never blocks the others.

Edge cases

  • Each result element is an independent ensure_result(): you can have a mix of {ok, N}, {ok, already} and {error, _} in the same list.
  • Empty list -> [] (no roundtrip to the server).
  • If ALL specs are idempotent/error in the preparation phase, no commit is sent (commit_many only runs when there is >=1 validated payload).
1> erli18n_server:ensure_loaded_many([
..    {my_domain, <<"fr">>, "fr.po", #{}},
..    {my_domain, <<"de">>, "de.po", #{}},
..    {my_domain, <<"xx">>, "/missing.po", #{}}
.. ]).
[{my_domain, <<"fr">>, {ok, 128}},
 {my_domain, <<"de">>, {ok, 96}},
 {my_domain, <<"xx">>, {error, {file_error, enoent}}}]

See ensure_loaded/4, load_spec().

handle_call/3

Serialized critical section — NOTE FOR THE MAINTAINER. This is the ONLY place where the ETS is mutated; every write in the module passes through here under the single mailbox, so there is never observable mixed state (ETS-set inserts atomically per row).

Message protocol (all call variants)

  • {insert_singular, D, L, Ctx, Msgid, T} -> writes 1 row + indexes the key; reply ok. (API: insert_singular/5.)
  • {insert_plural, D, L, Ctx, Msgid, Entries} -> writes 1 row per form + indexes; reply ok. (API: insert_plural/5.)
  • {insert_catalog, D, L, Entries} -> flattens and writes the batch + indexes; reply ok. (API: insert_catalog/3.)
  • {unload, D, L} -> O(catalog size) deletion via the index + removes the header; emits the span [erli18n, catalog, unload]; reply is ALWAYS ok (historical contract of unload/2).
  • {commit, ensure | reload, D, L, Staged} -> installs an ALREADY validated staged() (the heavy phase ran in the caller). Mode ensure RE-CHECKS idempotency under serialization (closes the check-then-insert race); mode reload does the atomic insert-before-prune swap (finding #4). There is NO span here — it already fired caller-side.

  • {commit_many, Items} -> installs N payloads in one critical section, with ONE memory_warning_check at the end (not N).
  • Any other call -> {reply, {error, unknown_call}, State}.

Invariant

The server receives ONLY validated payloads in the commits — no heavy read/parse/compile runs behind this mailbox (finding #6). That is why this callback is the millisecond section.

handle_cast(Msg, State)

Inert by design: this server uses no casts (every write is a synchronous call, so the caller gets acknowledgement and backpressure). It exists only to satisfy the gen_server behaviour. Messages are ignored with {noreply, State}.

handle_info(Info, State)

Inert by design: this server expects no out-of-band messages. The initial 'ETS-TRANSFER' is consumed in init/1 by an explicit receive, not here; a return 'ETS-TRANSFER' after a crash reaches the OWNER (heir), not this worker. Messages are ignored with {noreply, State}.

init/1

-spec init([]) -> {ok, map()}.

Initialization callback — NOTE FOR THE MAINTAINER (do not call by hand; it is the supervisor that invokes it via start_link/0).

Table acquisition protocol (heir / ETS-TRANSFER)

This server does NOT create the data table (finding #10). It asks erli18n_table_owner (a sibling started BEFORE it under rest_for_one) to hand it over via give_away/3. The sequence:

  1. erli18n_table_owner:claim_table/0 (synchronous) — signals the owner to hand over.
  2. Blocks on receive {'ETS-TRANSFER', ?ETS_TABLE, _, ?ETS_HANDOFF_DATA}.
  3. Creates the secondary index (?CATALOG_INDEX_TABLE) and REBUILDS it from the surviving rows of the data table (on initial boot it is a no-op; after a crash of this worker, the table comes back populated by the heir and the index is rebuilt ONCE here, not per load).

Invariants

  • The State is #{} (empty): all the truth lives in the two ETS tables. Do not keep catalog state in State.
  • This server is the SOLE writer of both tables, so the O(1) index never diverges while the worker is alive.

Failure modes

If the owner does not hand the table over within 5s, it crashes with {ets_handoff_timeout, _} — something structurally broken in the tree; the supervisor re-evaluates. After a crash of this worker, the catalogs are NOT lost (the owner is the heir).

See start_link/0.

insert_catalog(Domain, Locale, Entries)

-spec insert_catalog(domain(), locale(), [catalog_entry()]) -> ok.

Inserts a batch of entries (singular and plural) of a catalog in one write.

Parameters

  • Domain, Locale: the target catalog.
  • Entries: a list mixing {singular, Context, Msgid, Translation} and {plural, Context, Msgid, MsgidPlural, [{Index, Translation}]}. The MsgidPlural is preserved in the parsed format for dump/1 round-trips, but is discarded when materializing the ETS rows (it does not take part in the lookup key — finding #14).

Return and effects

Each entry is flattened into the corresponding ETS rows (one per plural form) and the resulting data keys are registered in the catalog index. Synchronous; always returns ok. Does NOT install the catalog header — for the full pipeline (.po parse + plural compile + header) use ensure_loaded/3. Without a header, lookup_plural_form/5 returns undefined; use this function for seeding singular data or in tests.

Failure modes

A non-atom Domain / non-binary Locale / non-list Entries crash with function_clause. An entry with an unknown tag crashes the server in entry_to_objects/3.

1> erli18n_server:insert_catalog(my_domain, <<"fr">>, [
..    {singular, undefined, <<"Hello">>, <<"Bonjour">>},
..    {plural, undefined, <<"file">>, <<"files">>, [{0, <<"fichier">>}, {1, <<"fichiers">>}]}
.. ]).
ok
2> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Hello">>).
{ok, <<"Bonjour">>}

See also insert_singular/5, insert_plural/5, ensure_loaded/3.

insert_plural(Domain, Locale, Context, Msgid, Entries)

-spec insert_plural(domain(), locale(), context(), msgid(), plural_entries()) -> ok.

Inserts/overwrites the plural forms of a Msgid, serialized by the server.

Parameters

  • Domain, Locale, Context, Msgid: as in insert_singular/5.
  • Entries: the list [{FormIndex, Translation}] — one plural form per pair, where FormIndex is the form index (0 = gettext singular, 1, 2, ...).

Return and effects

Writes one ETS row per form, under the key {plural, Domain, Locale, Context, Msgid, FormIndex} (the plural tag is part of the key — ?PLURAL_KEY in erli18n.hrl), and registers those keys in the catalog index. Synchronous; always returns ok. An empty list is a no-op: it writes no row AND does not register the catalog in the index — by the membership rule "index row present <=> >=1 data entry". Selecting the correct form at read time (evaluating Plural-Forms against N) is the responsibility of lookup_plural_form/5, NOT of this function: here you supply the raw indices.

Failure modes

Arguments outside the guards crash with function_clause. Each pair must have an integer FormIndex >= 0 (validated in build_plural_objects/5 inside the server); a negative index would crash the server.

The rows are immediately readable via a direct ETS read, but lookup_plural_form/5 only SELECTS a form when the (Domain, Locale) catalog header is already loaded — it reads the header first to obtain the Plural-Forms rule; without a header it returns undefined directly (header absent -> miss). And insert_plural/5 does NOT install any header.

Important: there is no shortcut to make a hand-inserted form "selectable" after the fact. ensure_loaded/3 does NOT install a retroactive header for keys written via insert_plural/5: it either takes the idempotent fast-path (if a header already exists) and touches nothing, or it stages and installs ONLY the entries read from the .po on disk (see do_ensure_loaded/4). For lookup_plural_form/5 to select a plural form, the header must have been installed by a real .po load that ALREADY contains those very keys. Without a .po, insert_plural/5 is only useful for seeding data to be read directly from ETS in tests, via the tagged key {plural, Domain, Locale, Context, Msgid, FormIndex}:

1> erli18n_server:insert_plural(my_domain, <<"fr">>, undefined, <<"file">>,
..    [{0, <<"fichier">>}, {1, <<"fichiers">>}]).
ok
%% Without a header, lookup_plural_form/5 is a miss:
2> erli18n_server:lookup_plural_form(my_domain, <<"fr">>, undefined, <<"file">>, 1).
undefined
%% In tests, read the row directly by the tagged key:
3> ets:lookup(erli18n_catalog, {plural, my_domain, <<"fr">>, undefined, <<"file">>, 1}).
[{{plural, my_domain, <<"fr">>, undefined, <<"file">>, 1}, <<"fichiers">>}]

See also insert_singular/5, lookup_plural_form/5, ensure_loaded/3.

insert_singular(Domain, Locale, Context, Msgid, Translation)

-spec insert_singular(domain(), locale(), context(), msgid(), translation()) -> ok.

Inserts/overwrites a singular translation, serialized by the server.

Low-level write API — to load entire .po files prefer ensure_loaded/3. Useful in tests or when feeding translations from a source other than .po.

Parameters

  • Domain: the catalog's gettext domain (atom).
  • Locale: the binary locale (e.g. <<"fr">>).
  • Context: the msgctxt, or undefined when absent.
  • Msgid: the source text (lookup key).
  • Translation: the translation to store.

Return and effects

Writes one ETS row under the key {singular, Domain, Locale, Context, Msgid} (the singular tag is part of the key — ?SINGULAR_KEY in erli18n.hrl) and registers that key in the catalog index (which thus starts counting >=1 entry). Synchronous; always returns ok. Overwrites any prior translation for the same key. Does NOT install a header — this entry is readable via lookup_singular/4, but the catalog will not have lookup_header/2 unless an ensure_loaded/3 installs it.

Failure modes

Arguments outside the guards (e.g. a non-atom Domain) crash with function_clause in the caller. A server reply other than ok crashes with badmatch (contract break — only handle_call/3 writes that reply).

1> erli18n_server:insert_singular(my_domain, <<"fr">>, undefined, <<"Hello">>, <<"Bonjour">>).
ok
2> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Hello">>).
{ok, <<"Bonjour">>}

See also insert_plural/5, insert_catalog/3, lookup_singular/4.

loaded_catalogs()

-spec loaded_catalogs() -> [{domain(), locale(), non_neg_integer()}].

Lists the loaded catalogs with the data-row count of each.

Returns [{Domain, Locale, NumRows}], where NumRows counts the data rows (singulars + EACH plural form counted separately; headers do NOT count — that is why this number differs from header_state().num_entries, which counts logical entries). The list order is unspecified.

Computed by scanning an ets:tab2list/1 snapshot (O(total rows)) — it is an observability read, not a hot path. For just the COUNT of catalogs without the scan cost, use memory_info/0 (num_catalogs, O(1)).

Relationship with memory_info/0: the NumRows here are ONLY data rows (no header), whereas memory_info/0.num_keys counts everything (data + 1 header per catalog). For the single catalog below, num_keys is 130 + 1 = 131.

1> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "fr.po").
{ok, 128}
2> erli18n_server:loaded_catalogs().
[{my_domain, <<"fr">>, 130}]

See also memory_info/0, which_keys/2.

lookup_header(Domain, Locale)

-spec lookup_header(domain(), locale()) -> {ok, header_state()} | undefined.

Lock-free lookup of the (Domain, Locale) catalog header, straight from ETS.

Return

  • {ok, HeaderState} — see header_state() for the contents (compiled plural rule or fallback, raw Plural-Forms, .po path, load instant, vs-CLDR divergence, entry count).
  • undefined if the catalog is not loaded.

Why this matters

The PRESENCE of the header is the idempotency signal ensure_loaded/3 consults ("already loaded?") and what lookup_plural_form/5 reads first to obtain the plural rule. A catalog loaded WITHOUT Plural-Forms still has a header — with plural := fallback. A catalog populated only by insert_* (without ensure_loaded/3) does NOT have a header.

Failure modes

A non-atom Domain / non-binary Locale crash with function_clause.

1> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "fr.po").
{ok, 128}
2> {ok, H} = erli18n_server:lookup_header(my_domain, <<"fr">>).
{ok, #{divergence => none,
       fuzzy_included => false,
       loaded_at => 1700000000000,
       num_entries => 128,
       plural => #{nplurals => 2, expr => '...', raw => <<"n>1">>},
       plural_raw => <<"nplurals=2; plural=n>1;">>,
       po_path => <<"fr.po">>}}
3> maps:get(num_entries, H).
128
4> erli18n_server:lookup_header(my_domain, <<"de">>).
undefined

(The plural value is a erli18n_plural:plural_compiled/0#{nplurals, expr, raw}; the expr above is abbreviated as '...', it is an internal AST.)

See also lookup_singular/4, lookup_plural_form/5, header_state().

lookup_plural_form(Domain, Locale, Context, Msgid, N)

-spec lookup_plural_form(domain(), locale(), context(), msgid(), integer()) ->
                            {ok, translation()} | undefined.

The CORRECT entry point for plural reads (form-aware, lock-free).

Unlike the raw, index-based lookup_plural/5 (internal, NOT exported — finding #16), here the caller does NOT need to know the form index for N: this function reads the header, evaluates the catalog's compiled Plural-Forms rule against the count N to obtain the form index, and then looks up the entry at that index. It is the encapsulation of the locale-specific knowledge the library exists to provide.

Parameters

  • Domain, Locale, Context, Msgid: identify the plural msgid.
  • N: the count (integer) that decides the form. NOT the form index — the Plural-Forms rule converts it into an index.

Return

  • {ok, Translation} when the form exists.
  • undefined on a miss — it is up to the caller to fall back to msgid_plural (PSD-003). There is no automatic fallback to the raw text here.

Hot path

1 header lookup + 1 entry lookup, both direct ets:lookup/2, with no roundtrip to the gen_server.

Fallback rules (order matters)

  • Header absent (catalog not loaded, or populated only by insert_*) -> undefined directly.
  • Header present without Plural-Forms (plural := fallback) -> uses the C/Germanic default (N == 1 -> form 0; otherwise form 1).
  • Header with a compiled rule -> evaluates the rule. erli18n_plural:evaluate/2 is total (it clamps malformed rules instead of crashing), and the surrounding try is belt-and-suspenders that degrades to the Germanic form should a future regression reintroduce a throw on this per-request hot path (finding #1, anti-DoS). In other words: this function never crashes the calling process because of an untrusted plural rule.

Failure modes

A non-integer N (or other args outside the guards) is function_clause.

%% French catalog loaded with Plural-Forms (n>1 -> form 1).
1> erli18n_server:lookup_plural_form(my_domain, <<"fr">>, undefined, <<"file">>, 1).
{ok, <<"fichier">>}
2> erli18n_server:lookup_plural_form(my_domain, <<"fr">>, undefined, <<"file">>, 42).
{ok, <<"fichiers">>}
3> erli18n_server:lookup_plural_form(my_domain, <<"de">>, undefined, <<"file">>, 1).
undefined

See also lookup_singular/4, lookup_header/2.

lookup_singular(Domain, Locale, Context, Msgid)

-spec lookup_singular(domain(), locale(), context(), msgid()) -> {ok, translation()} | undefined.

Lock-free lookup of a singular translation, straight from ETS in the caller.

The singular read hot path: a single ets:lookup/2, with no roundtrip to the gen_server, executed in the calling process (that is why N processes read in parallel with no bottleneck).

Parameters

  • Domain, Locale: the catalog.
  • Context: the msgctxt, or undefined. A lookup with the wrong Context is a miss — msgctxt is part of the key.
  • Msgid: the source text being looked up.

Return

  • {ok, Translation} if the key {singular, Domain, Locale, Context, Msgid} exists (the singular tag is part of the key — ?SINGULAR_KEY in erli18n.hrl; an ets:lookup/2 with the untagged 4-tuple is always a miss).
  • undefined on a miss — it is up to the caller (the erli18n façade) to apply the fallback to the raw Msgid itself. There is no automatic fallback here.

Failure modes

Arguments outside the guards are function_clause (contract break, LOUD failure), never a silent undefined — a deliberate choice (finding #16). If the ETS does not exist (dead server), ets:lookup/2 crashes with badarg.

1> erli18n_server:insert_singular(my_domain, <<"fr">>, undefined, <<"Hello">>, <<"Bonjour">>).
ok
2> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Hello">>).
{ok, <<"Bonjour">>}
3> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Missing">>).
undefined

See also lookup_plural_form/5, lookup_header/2.

memory_info()

-spec memory_info() ->
                     #{ets_bytes := non_neg_integer(),
                       num_catalogs := non_neg_integer(),
                       num_keys := non_neg_integer()}.

Returns the memory usage of the translations ETS table.

Observability read in the calling process; not a hot path (do not call it per request). Returns a map with:

  • ets_bytes: the data table's memory in bytes (words * wordsize).
  • num_catalogs: distinct loaded catalogs — read in O(1) from the secondary index (finding #7), not by scanning. Counts a catalog iff it has >=1 data row (a header-only .po does not count).
  • num_keys: total rows in the data table — ets:info(?ETS_TABLE, size), counts ALL rows, INCLUDING each catalog's header row. So, for a single catalog with 1 header, the invariant num_keys = (data rows from loaded_catalogs/0) + num_catalogs holds: the data rows (loaded_catalogs/0 does NOT count headers) plus one header row per catalog.

Failure modes

Crashes with {ets_info_invalid, _} if the table does not exist (dead server) — a descriptive payload instead of propagating undefined.

%% Single catalog with 130 data rows (see loaded_catalogs/0) + 1 header.
1> erli18n_server:memory_info().
#{ets_bytes => 24576, num_catalogs => 1, num_keys => 131}

See also loaded_catalogs/0, which_keys/2.

reload(Domain, Locale, PoPath)

-spec reload(domain(), locale(), file:filename()) -> ensure_result().

Atomic reload of a .po catalog. Same as reload/4 with #{}.

Unlike ensure_loaded/3, it NEVER takes the idempotent fast-path: it always parses and reinstalls, overwriting the old catalog entry by entry (AMB-001). It never returns {ok, already}. See reload/4 for the atomic STAGE -> SWAP semantics and the zero-miss-window guarantee.

1> erli18n_server:reload(my_domain, <<"fr">>, "fr.po").
{ok, 128}
%% An invalid .po does NOT destroy the good catalog in use:
2> erli18n_server:reload(my_domain, <<"fr">>, "broken.po").
{error, {parse_error, ...}}
3> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Hello">>).
{ok, <<"Bonjour">>}

See reload/4, ensure_loaded/3.

reload(Domain, Locale, PoPath, Opts)

-spec reload(domain(), locale(), file:filename(), opts()) -> ensure_result().

Atomic reload of a .po catalog with resource options (STAGE -> SWAP).

reload is an atomic STAGE -> SWAP. The entire failable half (read, parse, compile plural, CLDR divergence) runs in the CALLING process into an in-memory staged/0 WITHOUT touching the ETS, so a reload whose new .po is invalid (syntax error, unsupported charset, bad Plural-Forms, missing file) returns a structured {error, _} and leaves the previous catalog FULLY INTACT — never destroyed. On success, the only observable mutation is insert-before-prune: each retained key is overwritten old->new by an atomic ets:insert/2, and only the keys absent from the new catalog are pruned afterwards — a concurrent reader of a retained key never observes a miss window.

The heavy phase runs inside the span [erli18n, catalog, reload]; only the swap commit travels to the server, with a tunable timeout. Opts is identical to ensure_loaded/4's (include_fuzzy, max_bytes, max_entries, timeout). Returns {ok, NewlyLoaded} or {error, ensure_error()} (never {ok, already}).

Edge cases

  • Transient memory doubles during the swap (old + new rows coexist) — a deliberate cost of atomicity.
  • Stale keys (present in the old catalog, absent in the new) serve the OLD translation for microseconds before the prune — consistent with gettext, better than a miss.
  • The same bound_error()s as ensure_loaded/4 apply; on any error the previous catalog stays intact.
1> erli18n_server:reload(my_domain, <<"fr">>, "fr.po", #{timeout => 30000}).
{ok, 128}

See reload/3, ensure_loaded/4, opts().

start_link()

-spec start_link() -> gen_server:start_ret().

Starts the catalog gen_server, registered locally as erli18n_server.

Called by the supervisor — in general you do NOT call this by hand. In init/1 the server claims the data ETS table from erli18n_table_owner via 'ETS-TRANSFER' (becoming its owner/writer) and (re)builds the O(1) catalog index from the surviving rows.

Failure modes

init/1 crashes with {ets_handoff_timeout, _} if the owner does not hand the table over within 5s (something structurally broken in the supervision tree), which makes the supervisor re-evaluate. Since the owner is the table's heir, a crash of THIS worker does not destroy the catalogs: the table returns to the owner and is re-handed on the next init/1.

1> {ok, Pid} = erli18n_server:start_link().
{ok, <0.123.0>}
2> is_pid(Pid).
true

See also init/1.

terminate(Reason, State)

No cleanup to do: the data table is protected and has a heir (the owner), so on a crash of this worker it returns INTACT to the owner instead of being destroyed — exactly the scenario this architecture protects. The secondary index is server-owned and dies with the worker, being rebuilt on the next init/1. Returns ok.

unload(Domain, Locale)

-spec unload(domain(), locale()) -> ok.

Removes the (Domain, Locale) catalog entirely (entries + header).

Return and effects

O(catalog size) deletion, not O(total rows): it reads the data keys from the secondary index and deletes each one (O(1) per key on a set), then removes the header by its own key and deregisters the catalog from the index (finding #13). After the unload, lookup_* of that catalog starts returning undefined. Synchronous; always returns ok.

Idempotent: unloading a never-loaded catalog is a no-op (also returns ok). Emits the telemetry span [erli18n, catalog, unload] whose stop metadata includes result (ok | not_loaded) and keys_removed.

Failure modes

A non-atom Domain / non-binary Locale crash with function_clause.

1> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "fr.po").
{ok, 128}
2> erli18n_server:unload(my_domain, <<"fr">>).
ok
3> erli18n_server:lookup_header(my_domain, <<"fr">>).
undefined
4> erli18n_server:unload(my_domain, <<"fr">>).   %% idempotent
ok

See also reload/3 (which does NOT do delete-then-reload), loaded_catalogs/0.

which_keys(Domain, Locale)

-spec which_keys(domain(), locale()) -> [{singular, context(), msgid()} | {plural, context(), msgid()}].

Enumerates the keys (singular and plural) loaded for (Domain, Locale).

Returns a SORTED list of {singular, Context, Msgid} and {plural, Context, Msgid}. Plural entries are DEDUPLICATED by (Context, Msgid): a plural msgid with N forms appears ONCE, not N times (the loaded_catalogs/0 count, by contrast, counts each form). Parity with gettexter:which_keys/2.

ETS scan (ets:tab2list/1) in the calling process, with no roundtrip to the gen_server — observability, not a hot path. Absent catalog -> empty list.

1> erli18n_server:insert_singular(d, <<"fr">>, undefined, <<"Hello">>, <<"Bonjour">>).
ok
2> erli18n_server:insert_plural(d, <<"fr">>, undefined, <<"file">>,
..    [{0, <<"fichier">>}, {1, <<"fichiers">>}]).
ok
3> erli18n_server:which_keys(d, <<"fr">>).
[{plural, undefined, <<"file">>}, {singular, undefined, <<"Hello">>}]

See also loaded_catalogs/0, memory_info/0.