Catalog gen_server: owner and sole writer of the translations ETS table.
What it is and which problem it solves
This module is the heart of the erli18n runtime: it loads .po catalogs,
keeps the translations live in ETS and answers lookups (singular, plural and
header). The central problem it solves is reconciling two contradictory
requirements: translation reads are extremely hot (every UI string goes
through a lookup) and must be lock-free; writes, on the other hand, must be
serialized to keep the protected ETS consistent. The solution is a strict
split between the write path (serialized by this process's mailbox) and the
read path (straight from ETS, with no roundtrip to the server).
Mental model
Think of this module in three layers:
Read path (hot path, lock-free).
lookup_singular/4,lookup_plural_form/5andlookup_header/2runets:lookup/2directly in the CALLING process — no message reaches the server mailbox. That is why N processes can read in parallel with no bottleneck (RISK-012). The ETS isprotected: any process reads, only the owner (this server) writes.Write path (serialized).
insert_*,unload/2and the load commits becomegen_server:call/2,3;handle_call/3is the only critical section where the ETS is mutated. Since ETS-setperforms atomic per-row inserts under the single mailbox, there is never observable mixed state.Load orchestration (heavy work OUTSIDE the mailbox).
ensure_loaded/4andreload/4run the heavy, failable phase (size-check, read, parse, compile of the plural rule, CLDR divergence, object build, fuzzy count) in the CALLING process, producing a pure in-memorystaged(). Only this validated payload travels to the server for a millisecond-scale commit. This way a large/slow/pathological.pofrom one tenant never blocks another's load (finding #6).
Trusted vs untrusted. A .po's Plural-Forms rule is untrusted input;
that is why it is compiled with bounds (see erli18n_plural) and
lookup_plural_form/5 wraps the evaluation in a belt-and-suspenders try. This
module's anti-DoS bounds (max_bytes, max_entries) reject large catalogs
BEFORE any ETS mutation.
ETS-TRANSFER / heir. This server does NOT create the data table. A
dedicated owner (erli18n_table_owner, started before it under rest_for_one)
creates the table and hands it over via give_away/3 ('ETS-TRANSFER'
consumed in init/1), remaining as heir. If this worker crashes, the table
returns INTACT to the owner instead of being destroyed — every loaded catalog
survives the restart. On reclaiming it, init/1 rebuilds the O(1) catalog
index from the surviving rows.
Two mental maps of "state". This gen_server's State is an empty map
#{} — that is deliberate. The truth lives in TWO ETS tables: the data table
(?ETS_TABLE, inherited from the owner) and a server-owned secondary index
(?CATALOG_INDEX_TABLE) that stores, per catalog, the set of data keys.
The index makes memory_info/0 O(1) and unload/2 O(catalog size) instead of
O(total rows) scans (findings #7/#13).
When and how a dev touches this module
- To load/reload a
.po:ensure_loaded/3,4(idempotent),reload/3,4(always reinstalls, atomic) orensure_loaded_many/1(batch). - To unload:
unload/2. - To read a low-level translation (the
erli18nfaçade is the usual front-door):lookup_singular/4,lookup_plural_form/5,lookup_header/2. - To write individual entries (tests, non-
.posources):insert_singular/5,insert_plural/5,insert_catalog/3. - For observability:
memory_info/0,loaded_catalogs/0,which_keys/2.
Quickstart
1> application:ensure_all_started(erli18n).
{ok, [erli18n]}
2> Po = erli18n_server:default_po_path(my_app, my_domain, <<"fr">>).
"/.../priv/locale/fr/LC_MESSAGES/my_domain.po"
3> erli18n_server:ensure_loaded(my_domain, <<"fr">>, Po).
{ok, 128}
4> erli18n_server:ensure_loaded(my_domain, <<"fr">>, Po).
{ok, already}
5> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Hello">>).
{ok, <<"Bonjour">>}
6> erli18n_server:lookup_plural_form(my_domain, <<"fr">>, undefined, <<"file">>, 2).
{ok, <<"fichiers">>}
7> erli18n_server:memory_info().
#{ets_bytes => 24576, num_catalogs => 1, num_keys => 131}(num_keys counts ALL rows, including the header; loaded_catalogs/0 counts
only the 130 data rows — see both functions.)
Main entry points
- Load:
ensure_loaded/3,ensure_loaded/4,ensure_loaded_many/1,reload/3,reload/4. - Read (lock-free):
lookup_singular/4,lookup_plural_form/5,lookup_header/2. - Write:
insert_singular/5,insert_plural/5,insert_catalog/3,unload/2. - Observability:
memory_info/0,loaded_catalogs/0,which_keys/2. - Lifecycle / OTP:
start_link/0,init/1.
Summary
Types
Errors from the anti-DoS bounds (finding #6), a subset of ensure_error().
Both are surfaced in the CALLER's heavy phase, BEFORE any ETS mutation:
input_too_large when the file size exceeds max_bytes (checked without
reading the bytes, via filelib:file_size/1); too_many_entries when the
post-parse count exceeds max_entries. The second element is the observed
value, the third is the configured limit.
Union of all structured errors a load can return. Each variant maps a failable
step of the stage pipeline (in the order it can fail): an I/O error reading the
file ({file_error, _}), a .po parse error (erli18n_po:parse_error()), a
plural-rule compile error ({plural_compile_error, _}) and the anti-DoS caps
(bound_error()). None of them leaves the ETS mutated.
Result of a load (ensure_loaded/3,4, reload/3,4).
Header state of a loaded catalog, returned by lookup_header/2 and stored in
the ?HEADER_KEY row. The PRESENCE of this row is the idempotency signal used
by ensure_loaded/3 ("catalog already loaded").
A catalog to load in the bulk API ensure_loaded_many/1. Same positional
shape as the ensure_loaded/4 arguments:
{Domain, Locale, PoPath, Opts}.
Load options accepted by ensure_loaded/4, reload/4 and each item of
ensure_loaded_many/1. All fields are optional; omitting one preserves the
legacy behaviour (modulo the safety-cap defaults).
Functions
No state migration: the State is an empty #{} (the truth lives in ETS), so a
code upgrade has no state to transform. Returns {ok, State}.
Computes the conventional gettext .po path for an application.
Idempotent load of a .po catalog. Same as ensure_loaded/4 with #{}.
Idempotent load of a .po catalog with resource options.
Bulk load of N catalogs with a single commit on the server.
Serialized critical section — NOTE FOR THE MAINTAINER. This is the ONLY place
where the ETS is mutated; every write in the module passes through here under the
single mailbox, so there is never observable mixed state (ETS-set inserts
atomically per row).
Inert by design: this server uses no casts (every write is a synchronous call,
so the caller gets acknowledgement and backpressure). It exists only to satisfy
the gen_server behaviour. Messages are ignored with {noreply, State}.
Inert by design: this server expects no out-of-band messages. The initial
'ETS-TRANSFER' is consumed in init/1 by an explicit receive, not here; a
return 'ETS-TRANSFER' after a crash reaches the OWNER (heir), not this worker.
Messages are ignored with {noreply, State}.
Initialization callback — NOTE FOR THE MAINTAINER (do not call by hand; it is
the supervisor that invokes it via start_link/0).
Inserts a batch of entries (singular and plural) of a catalog in one write.
Inserts/overwrites the plural forms of a Msgid, serialized by the server.
Inserts/overwrites a singular translation, serialized by the server.
Lists the loaded catalogs with the data-row count of each.
Lock-free lookup of the (Domain, Locale) catalog header, straight from ETS.
The CORRECT entry point for plural reads (form-aware, lock-free).
Lock-free lookup of a singular translation, straight from ETS in the caller.
Returns the memory usage of the translations ETS table.
Atomic reload of a .po catalog. Same as reload/4 with #{}.
Atomic reload of a .po catalog with resource options (STAGE -> SWAP).
Starts the catalog gen_server, registered locally as erli18n_server.
No cleanup to do: the data table is protected and has a heir (the owner), so
on a crash of this worker it returns INTACT to the owner instead of being
destroyed — exactly the scenario this architecture protects. The secondary index
is server-owned and dies with the worker, being rebuilt on the next init/1.
Returns ok.
Removes the (Domain, Locale) catalog entirely (entries + header).
Enumerates the keys (singular and plural) loaded for (Domain, Locale).
Types
-type bound_error() :: {input_too_large, Bytes :: non_neg_integer(), Limit :: non_neg_integer()} | {too_many_entries, Count :: non_neg_integer(), Limit :: non_neg_integer()}.
Errors from the anti-DoS bounds (finding #6), a subset of ensure_error().
Both are surfaced in the CALLER's heavy phase, BEFORE any ETS mutation:
input_too_large when the file size exceeds max_bytes (checked without
reading the bytes, via filelib:file_size/1); too_many_entries when the
post-parse count exceeds max_entries. The second element is the observed
value, the third is the configured limit.
-type catalog_entry() :: singular_entry() | plural_entry().
-type context() :: undefined | binary().
-type domain() :: atom().
-type ensure_error() :: erli18n_po:parse_error() | {plural_compile_error, erli18n_plural:compile_error()} | {file_error, file:posix() | badarg | terminated | system_limit} | bound_error() | {load_failed, term()}.
Union of all structured errors a load can return. Each variant maps a failable
step of the stage pipeline (in the order it can fail): an I/O error reading the
file ({file_error, _}), a .po parse error (erli18n_po:parse_error()), a
plural-rule compile error ({plural_compile_error, _}) and the anti-DoS caps
(bound_error()). None of them leaves the ETS mutated.
-type ensure_result() :: {ok, NewlyLoaded :: non_neg_integer()} | {ok, already} | {error, ensure_error()}.
Result of a load (ensure_loaded/3,4, reload/3,4).
{ok, NewlyLoaded}: a real load — number of entries parsed, compiled and installed.{ok, already}: idempotent fast-path, the catalog was already loaded (onlyensure_loaded/ensure_loaded_many;reloadnever returns this).{error, ensure_error()}: structured error; the ETS stays intact (all errors occur BEFORE any mutation).
-type header_state() :: #{plural := erli18n_plural:plural_compiled() | fallback, plural_raw := binary(), po_path := file:filename(), loaded_at := integer(), divergence := divergence_info(), fuzzy_included := boolean(), num_entries := non_neg_integer()}.
Header state of a loaded catalog, returned by lookup_header/2 and stored in
the ?HEADER_KEY row. The PRESENCE of this row is the idempotency signal used
by ensure_loaded/3 ("catalog already loaded").
plural: the ALREADY compiledPlural-Formsrule (erli18n_plural:plural_compiled()), or the atomfallbackwhen the.pocame without a plural header (the lookup then uses the C/Germanic default, seelookup_plural_form/5).plural_raw: the raw text of the rule (or the fallback rule) for observability/round-trip.po_path: the path of the source.po.loaded_at:erlang:system_time(millisecond)at the moment of the load.divergence:divergence_info()—noneor the vs-CLDR divergence warning.fuzzy_included: whether the load included#, fuzzyentries.num_entries: entry count (singular + plural aggregated as the parser counts them), the number reported in{ok, NewlyLoaded}.
-type load_spec() :: {domain(), locale(), file:filename(), opts()}.
A catalog to load in the bulk API ensure_loaded_many/1. Same positional
shape as the ensure_loaded/4 arguments:
{Domain, Locale, PoPath, Opts}.
-type locale() :: binary().
-type msgid() :: binary().
-type msgid_plural() :: undefined | binary().
-type opts() :: #{include_fuzzy => boolean(), max_bytes => non_neg_integer() | infinity, max_entries => non_neg_integer() | infinity, timeout => timeout()}.
Load options accepted by ensure_loaded/4, reload/4 and each item of
ensure_loaded_many/1. All fields are optional; omitting one preserves the
legacy behaviour (modulo the safety-cap defaults).
include_fuzzy(defaultfalse): includes entries marked#, fuzzy.max_bytes(defaultapplication:get_env(erli18n, max_po_bytes), 16 MiB): rejects the file BEFORE reading it whole (viafilelib:file_size/1).infinitydisables the cap.max_entries(defaultapplication:get_env(erli18n, max_po_entries), 500000): rejects the catalog AFTER the parse if it has more than N entries.infinitydisables the cap.timeout(default 5000 ms): deadline of thegen_server:call/3that performs the commit. Since the heavy phase no longer runs behind the mailbox, the deadline covers only the bulk insert (millisecond scale).
-type plural_entries() :: [{plural_index(), translation()}].
-type plural_entry() :: {plural, context(), msgid(), msgid_plural(), plural_entries()}.
-type plural_index() :: non_neg_integer().
-type singular_entry() :: {singular, context(), msgid(), translation()}.
-type translation() :: binary().
Functions
No state migration: the State is an empty #{} (the truth lives in ETS), so a
code upgrade has no state to transform. Returns {ok, State}.
-spec default_po_path(atom(), domain(), locale()) -> file:filename().
Computes the conventional gettext .po path for an application.
Parameters
App: the OTP application whoseprivcontains the catalogs (resolved viacode:priv_dir/1).Domain: the gettext domain (becomes the file name<Domain>.po).Locale: the binary locale (becomes the directory segment<Locale>).
Return
Returns <priv>/locale/<Locale>/LC_MESSAGES/<Domain>.po (a string).
This function only COMPOSES the path — it does not check whether the file exists.
Failure modes
Crashes with {priv_dir_not_found, App} if the application is unknown
(code:priv_dir/1 returns {error, bad_name}) — so the operator sees the
misconfiguration immediately, instead of loading a path with {error, bad_name}
embedded. Separation of concerns: ensure_loaded/3 takes the path explicitly;
there is no implicit resolution here — it is the façade that decides whether to
honour this convention or use a caller-supplied path.
1> erli18n_server:default_po_path(my_app, my_domain, <<"fr">>).
"/path/to/my_app/priv/locale/fr/LC_MESSAGES/my_domain.po"See ensure_loaded/3.
-spec ensure_loaded(domain(), locale(), file:filename()) -> ensure_result().
Idempotent load of a .po catalog. Same as ensure_loaded/4 with #{}.
If (Domain, Locale) is already loaded (header present), returns
{ok, already} without touching disk. Otherwise it runs the full pipeline (read
file, parse, compile the plural rule, validate against CLDR as a WARNING —
divergence never blocks the load, install in ETS atomically) and returns
{ok, NewlyLoaded} with the number of inserted entries, or
{error, ensure_error()} leaving the ETS INTACT.
1> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "priv/locale/fr/LC_MESSAGES/my_domain.po").
{ok, 128}
2> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "priv/locale/fr/LC_MESSAGES/my_domain.po").
{ok, already}
3> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "/no/such/file.po").
{error, {file_error, enoent}}See ensure_loaded/4 (options and bounds), reload/3 (forces reinstall),
ensure_loaded_many/1 (batch).
-spec ensure_loaded(domain(), locale(), file:filename(), opts()) -> ensure_result().
Idempotent load of a .po catalog with resource options.
Idempotent fast-path: if the catalog is already loaded, returns {ok, already}
via a pure ETS read (no disk, no server roundtrip). On a miss, the heavy phase
(read+parse+compile+validate+bounds) runs in the CALLING process, inside the
span [erli18n, catalog, load], and only the validated payload is handed to the
server for the millisecond commit.
Opts (all optional):
include_fuzzy(defaultfalse): includes entries marked#, fuzzy.max_bytes(non_neg_integer() | infinity): rejects the file BEFORE reading it whole (viafilelib:file_size/1) if it exceeds the limit; default comes fromapplication:get_env(erli18n, max_po_bytes)(16 MiB).infinity= no cap.max_entries(non_neg_integer() | infinity): rejects the catalog AFTER the parse if it has more than N entries; defaultapplication:get_env(erli18n, max_po_entries)(500000).infinity= no cap.timeout(timeout()): deadline of the commitgen_server:call/3(default 5000 ms; the commit is only the bulk insert, ~26 ms for 40k entries).
Returns {ok, NewlyLoaded}, {ok, already} or {error, ensure_error()}
(including {input_too_large, _, _} / {too_many_entries, _, _}), always before
any ETS mutation.
Edge cases
- Check-then-insert race: the idempotent fast-path reads outside the
serialization, but the commit RE-CHECKS idempotency under the mailbox (mode
ensure), so two concurrent callers of the same catalog do not overwrite each other — the second sees{ok, already}. - CLDR divergence: never an error; emits a warning log/telemetry and
proceeds, storing the divergence in the
header_state(). timeoutexceeded: the commitgen_server:call/3may crash with{timeout, _}; since the commit is only the bulk insert, this practically only happens with a very tighttimeout.
1> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "fr.po", #{include_fuzzy => true}).
{ok, 131}
2> erli18n_server:ensure_loaded(my_domain, <<"de">>, "big.po", #{max_bytes => 1024}).
{error, {input_too_large, 6553600, 1024}}See ensure_loaded/3, reload/4, ensure_loaded_many/1, opts(),
ensure_error().
-spec ensure_loaded_many([load_spec()]) -> [{domain(), locale(), ensure_result()}].
Bulk load of N catalogs with a single commit on the server.
Specs is [{Domain, Locale, PoPath, Opts}]. The heavy phase of each spec runs
in the calling process (sequential preparation — the v0.1 trade-off; a parallel
fan-out is a future evolution) and all ready payloads are delivered in a SINGLE
commit, collapsing N roundtrips into one and making the per-catalog idempotency
check O(1) (finding #7's O(1) index). Each Opts follows ensure_loaded/4.
Returns [{Domain, Locale, ensure_result()}] — already-loaded or failed catalogs
are reported individually; one catalog's error never blocks the others.
Edge cases
- Each result element is an independent
ensure_result(): you can have a mix of{ok, N},{ok, already}and{error, _}in the same list. - Empty list ->
[](no roundtrip to the server). - If ALL specs are idempotent/error in the preparation phase, no commit is sent
(
commit_manyonly runs when there is >=1 validated payload).
1> erli18n_server:ensure_loaded_many([
.. {my_domain, <<"fr">>, "fr.po", #{}},
.. {my_domain, <<"de">>, "de.po", #{}},
.. {my_domain, <<"xx">>, "/missing.po", #{}}
.. ]).
[{my_domain, <<"fr">>, {ok, 128}},
{my_domain, <<"de">>, {ok, 96}},
{my_domain, <<"xx">>, {error, {file_error, enoent}}}]See ensure_loaded/4, load_spec().
Serialized critical section — NOTE FOR THE MAINTAINER. This is the ONLY place
where the ETS is mutated; every write in the module passes through here under the
single mailbox, so there is never observable mixed state (ETS-set inserts
atomically per row).
Message protocol (all call variants)
{insert_singular, D, L, Ctx, Msgid, T}-> writes 1 row + indexes the key; replyok. (API:insert_singular/5.){insert_plural, D, L, Ctx, Msgid, Entries}-> writes 1 row per form + indexes; replyok. (API:insert_plural/5.){insert_catalog, D, L, Entries}-> flattens and writes the batch + indexes; replyok. (API:insert_catalog/3.){unload, D, L}-> O(catalog size) deletion via the index + removes the header; emits the span[erli18n, catalog, unload]; reply is ALWAYSok(historical contract ofunload/2).{commit, ensure | reload, D, L, Staged}-> installs an ALREADY validatedstaged()(the heavy phase ran in the caller). ModeensureRE-CHECKS idempotency under serialization (closes the check-then-insert race); modereloaddoes the atomic insert-before-prune swap (finding #4). There is NO span here — it already fired caller-side.{commit_many, Items}-> installs N payloads in one critical section, with ONEmemory_warning_checkat the end (not N).- Any other call ->
{reply, {error, unknown_call}, State}.
Invariant
The server receives ONLY validated payloads in the commits — no heavy read/parse/compile runs behind this mailbox (finding #6). That is why this callback is the millisecond section.
Inert by design: this server uses no casts (every write is a synchronous call,
so the caller gets acknowledgement and backpressure). It exists only to satisfy
the gen_server behaviour. Messages are ignored with {noreply, State}.
Inert by design: this server expects no out-of-band messages. The initial
'ETS-TRANSFER' is consumed in init/1 by an explicit receive, not here; a
return 'ETS-TRANSFER' after a crash reaches the OWNER (heir), not this worker.
Messages are ignored with {noreply, State}.
-spec init([]) -> {ok, map()}.
Initialization callback — NOTE FOR THE MAINTAINER (do not call by hand; it is
the supervisor that invokes it via start_link/0).
Table acquisition protocol (heir / ETS-TRANSFER)
This server does NOT create the data table (finding #10). It asks
erli18n_table_owner (a sibling started BEFORE it under rest_for_one) to hand
it over via give_away/3. The sequence:
erli18n_table_owner:claim_table/0(synchronous) — signals the owner to hand over.- Blocks on
receive {'ETS-TRANSFER', ?ETS_TABLE, _, ?ETS_HANDOFF_DATA}. - Creates the secondary index (
?CATALOG_INDEX_TABLE) and REBUILDS it from the surviving rows of the data table (on initial boot it is a no-op; after a crash of this worker, the table comes back populated by the heir and the index is rebuilt ONCE here, not per load).
Invariants
- The
Stateis#{}(empty): all the truth lives in the two ETS tables. Do not keep catalog state inState. - This server is the SOLE writer of both tables, so the O(1) index never diverges while the worker is alive.
Failure modes
If the owner does not hand the table over within 5s, it crashes with
{ets_handoff_timeout, _} — something structurally broken in the tree; the
supervisor re-evaluates. After a crash of this worker, the catalogs are NOT lost
(the owner is the heir).
See start_link/0.
-spec insert_catalog(domain(), locale(), [catalog_entry()]) -> ok.
Inserts a batch of entries (singular and plural) of a catalog in one write.
Parameters
Domain,Locale: the target catalog.Entries: a list mixing{singular, Context, Msgid, Translation}and{plural, Context, Msgid, MsgidPlural, [{Index, Translation}]}. TheMsgidPluralis preserved in the parsed format fordump/1round-trips, but is discarded when materializing the ETS rows (it does not take part in the lookup key — finding #14).
Return and effects
Each entry is flattened into the corresponding ETS rows (one per plural form) and
the resulting data keys are registered in the catalog index. Synchronous; always
returns ok. Does NOT install the catalog header — for the full pipeline
(.po parse + plural compile + header) use ensure_loaded/3. Without a header,
lookup_plural_form/5 returns undefined; use this function for seeding
singular data or in tests.
Failure modes
A non-atom Domain / non-binary Locale / non-list Entries crash with
function_clause. An entry with an unknown tag crashes the server in
entry_to_objects/3.
1> erli18n_server:insert_catalog(my_domain, <<"fr">>, [
.. {singular, undefined, <<"Hello">>, <<"Bonjour">>},
.. {plural, undefined, <<"file">>, <<"files">>, [{0, <<"fichier">>}, {1, <<"fichiers">>}]}
.. ]).
ok
2> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Hello">>).
{ok, <<"Bonjour">>}See also insert_singular/5, insert_plural/5, ensure_loaded/3.
-spec insert_plural(domain(), locale(), context(), msgid(), plural_entries()) -> ok.
Inserts/overwrites the plural forms of a Msgid, serialized by the server.
Parameters
Domain,Locale,Context,Msgid: as ininsert_singular/5.Entries: the list[{FormIndex, Translation}]— one plural form per pair, whereFormIndexis the form index (0 = gettext singular, 1, 2, ...).
Return and effects
Writes one ETS row per form, under the key
{plural, Domain, Locale, Context, Msgid, FormIndex} (the plural tag is part
of the key — ?PLURAL_KEY in erli18n.hrl), and registers those keys in the
catalog index. Synchronous; always returns ok. An empty list is a no-op:
it writes no row AND does not register the catalog in the index — by the
membership rule "index row present <=> >=1 data entry". Selecting the correct
form at read time (evaluating Plural-Forms against N) is the responsibility
of lookup_plural_form/5, NOT of this function: here you supply the raw indices.
Failure modes
Arguments outside the guards crash with function_clause. Each pair must have an
integer FormIndex >= 0 (validated in build_plural_objects/5 inside the
server); a negative index would crash the server.
The rows are immediately readable via a direct ETS read, but
lookup_plural_form/5 only SELECTS a form when the (Domain, Locale) catalog
header is already loaded — it reads the header first to obtain the Plural-Forms
rule; without a header it returns undefined directly (header absent -> miss).
And insert_plural/5 does NOT install any header.
Important: there is no shortcut to make a hand-inserted form "selectable" after
the fact. ensure_loaded/3 does NOT install a retroactive header for keys
written via insert_plural/5: it either takes the idempotent fast-path (if a
header already exists) and touches nothing, or it stages and installs ONLY the
entries read from the .po on disk (see do_ensure_loaded/4). For
lookup_plural_form/5 to select a plural form, the header must have been
installed by a real .po load that ALREADY contains those very keys. Without a
.po, insert_plural/5 is only useful for seeding data to be read directly from
ETS in tests, via the tagged key
{plural, Domain, Locale, Context, Msgid, FormIndex}:
1> erli18n_server:insert_plural(my_domain, <<"fr">>, undefined, <<"file">>,
.. [{0, <<"fichier">>}, {1, <<"fichiers">>}]).
ok
%% Without a header, lookup_plural_form/5 is a miss:
2> erli18n_server:lookup_plural_form(my_domain, <<"fr">>, undefined, <<"file">>, 1).
undefined
%% In tests, read the row directly by the tagged key:
3> ets:lookup(erli18n_catalog, {plural, my_domain, <<"fr">>, undefined, <<"file">>, 1}).
[{{plural, my_domain, <<"fr">>, undefined, <<"file">>, 1}, <<"fichiers">>}]See also insert_singular/5, lookup_plural_form/5, ensure_loaded/3.
-spec insert_singular(domain(), locale(), context(), msgid(), translation()) -> ok.
Inserts/overwrites a singular translation, serialized by the server.
Low-level write API — to load entire .po files prefer ensure_loaded/3.
Useful in tests or when feeding translations from a source other than .po.
Parameters
Domain: the catalog's gettext domain (atom).Locale: the binary locale (e.g.<<"fr">>).Context: themsgctxt, orundefinedwhen absent.Msgid: the source text (lookup key).Translation: the translation to store.
Return and effects
Writes one ETS row under the key {singular, Domain, Locale, Context, Msgid}
(the singular tag is part of the key — ?SINGULAR_KEY in erli18n.hrl) and
registers that key in the catalog index (which thus starts counting >=1 entry).
Synchronous; always returns ok. Overwrites any prior translation for the same
key. Does NOT install a header — this entry is readable via lookup_singular/4,
but the catalog will not have lookup_header/2 unless an ensure_loaded/3
installs it.
Failure modes
Arguments outside the guards (e.g. a non-atom Domain) crash with
function_clause in the caller. A server reply other than ok crashes with
badmatch (contract break — only handle_call/3 writes that reply).
1> erli18n_server:insert_singular(my_domain, <<"fr">>, undefined, <<"Hello">>, <<"Bonjour">>).
ok
2> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Hello">>).
{ok, <<"Bonjour">>}See also insert_plural/5, insert_catalog/3, lookup_singular/4.
-spec loaded_catalogs() -> [{domain(), locale(), non_neg_integer()}].
Lists the loaded catalogs with the data-row count of each.
Returns [{Domain, Locale, NumRows}], where NumRows counts the data rows
(singulars + EACH plural form counted separately; headers do NOT count — that is
why this number differs from header_state().num_entries, which counts logical
entries). The list order is unspecified.
Computed by scanning an ets:tab2list/1 snapshot (O(total rows)) — it is an
observability read, not a hot path. For just the COUNT of catalogs without the
scan cost, use memory_info/0 (num_catalogs, O(1)).
Relationship with memory_info/0: the NumRows here are ONLY data rows (no
header), whereas memory_info/0.num_keys counts everything (data + 1 header
per catalog). For the single catalog below, num_keys is 130 + 1 = 131.
1> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "fr.po").
{ok, 128}
2> erli18n_server:loaded_catalogs().
[{my_domain, <<"fr">>, 130}]See also memory_info/0, which_keys/2.
-spec lookup_header(domain(), locale()) -> {ok, header_state()} | undefined.
Lock-free lookup of the (Domain, Locale) catalog header, straight from ETS.
Return
{ok, HeaderState}— seeheader_state()for the contents (compiled plural rule orfallback, rawPlural-Forms,.popath, load instant, vs-CLDR divergence, entry count).undefinedif the catalog is not loaded.
Why this matters
The PRESENCE of the header is the idempotency signal ensure_loaded/3 consults
("already loaded?") and what lookup_plural_form/5 reads first to obtain the
plural rule. A catalog loaded WITHOUT Plural-Forms still has a header — with
plural := fallback. A catalog populated only by insert_* (without
ensure_loaded/3) does NOT have a header.
Failure modes
A non-atom Domain / non-binary Locale crash with function_clause.
1> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "fr.po").
{ok, 128}
2> {ok, H} = erli18n_server:lookup_header(my_domain, <<"fr">>).
{ok, #{divergence => none,
fuzzy_included => false,
loaded_at => 1700000000000,
num_entries => 128,
plural => #{nplurals => 2, expr => '...', raw => <<"n>1">>},
plural_raw => <<"nplurals=2; plural=n>1;">>,
po_path => <<"fr.po">>}}
3> maps:get(num_entries, H).
128
4> erli18n_server:lookup_header(my_domain, <<"de">>).
undefined(The plural value is a erli18n_plural:plural_compiled/0 —
#{nplurals, expr, raw}; the expr above is abbreviated as '...', it is an
internal AST.)
See also lookup_singular/4, lookup_plural_form/5, header_state().
-spec lookup_plural_form(domain(), locale(), context(), msgid(), integer()) -> {ok, translation()} | undefined.
The CORRECT entry point for plural reads (form-aware, lock-free).
Unlike the raw, index-based lookup_plural/5 (internal, NOT exported — finding
#16), here the caller does NOT need to know the form index for N: this
function reads the header, evaluates the catalog's compiled Plural-Forms rule
against the count N to obtain the form index, and then looks up the entry at
that index. It is the encapsulation of the locale-specific knowledge the library
exists to provide.
Parameters
Domain,Locale,Context,Msgid: identify the plural msgid.N: the count (integer) that decides the form. NOT the form index — thePlural-Formsrule converts it into an index.
Return
{ok, Translation}when the form exists.undefinedon a miss — it is up to the caller to fall back tomsgid_plural(PSD-003). There is no automatic fallback to the raw text here.
Hot path
1 header lookup + 1 entry lookup, both direct ets:lookup/2, with no roundtrip
to the gen_server.
Fallback rules (order matters)
- Header absent (catalog not loaded, or populated only by
insert_*) ->undefineddirectly. - Header present without
Plural-Forms(plural := fallback) -> uses the C/Germanic default (N == 1 -> form 0; otherwise form 1). - Header with a compiled rule -> evaluates the rule.
erli18n_plural:evaluate/2is total (it clamps malformed rules instead of crashing), and the surroundingtryis belt-and-suspenders that degrades to the Germanic form should a future regression reintroduce a throw on this per-request hot path (finding #1, anti-DoS). In other words: this function never crashes the calling process because of an untrusted plural rule.
Failure modes
A non-integer N (or other args outside the guards) is function_clause.
%% French catalog loaded with Plural-Forms (n>1 -> form 1).
1> erli18n_server:lookup_plural_form(my_domain, <<"fr">>, undefined, <<"file">>, 1).
{ok, <<"fichier">>}
2> erli18n_server:lookup_plural_form(my_domain, <<"fr">>, undefined, <<"file">>, 42).
{ok, <<"fichiers">>}
3> erli18n_server:lookup_plural_form(my_domain, <<"de">>, undefined, <<"file">>, 1).
undefinedSee also lookup_singular/4, lookup_header/2.
-spec lookup_singular(domain(), locale(), context(), msgid()) -> {ok, translation()} | undefined.
Lock-free lookup of a singular translation, straight from ETS in the caller.
The singular read hot path: a single ets:lookup/2, with no roundtrip to the
gen_server, executed in the calling process (that is why N processes read in
parallel with no bottleneck).
Parameters
Domain,Locale: the catalog.Context: themsgctxt, orundefined. A lookup with the wrongContextis a miss —msgctxtis part of the key.Msgid: the source text being looked up.
Return
{ok, Translation}if the key{singular, Domain, Locale, Context, Msgid}exists (thesingulartag is part of the key —?SINGULAR_KEYinerli18n.hrl; anets:lookup/2with the untagged 4-tuple is always a miss).undefinedon a miss — it is up to the caller (theerli18nfaçade) to apply the fallback to the rawMsgiditself. There is no automatic fallback here.
Failure modes
Arguments outside the guards are function_clause (contract break, LOUD
failure), never a silent undefined — a deliberate choice (finding #16). If the
ETS does not exist (dead server), ets:lookup/2 crashes with badarg.
1> erli18n_server:insert_singular(my_domain, <<"fr">>, undefined, <<"Hello">>, <<"Bonjour">>).
ok
2> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Hello">>).
{ok, <<"Bonjour">>}
3> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Missing">>).
undefinedSee also lookup_plural_form/5, lookup_header/2.
-spec memory_info() -> #{ets_bytes := non_neg_integer(), num_catalogs := non_neg_integer(), num_keys := non_neg_integer()}.
Returns the memory usage of the translations ETS table.
Observability read in the calling process; not a hot path (do not call it per request). Returns a map with:
ets_bytes: the data table's memory in bytes (words * wordsize).num_catalogs: distinct loaded catalogs — read in O(1) from the secondary index (finding #7), not by scanning. Counts a catalog iff it has >=1 data row (a header-only.podoes not count).num_keys: total rows in the data table —ets:info(?ETS_TABLE, size), counts ALL rows, INCLUDING each catalog's header row. So, for a single catalog with 1 header, the invariantnum_keys = (data rows from loaded_catalogs/0) + num_catalogsholds: the data rows (loaded_catalogs/0does NOT count headers) plus one header row per catalog.
Failure modes
Crashes with {ets_info_invalid, _} if the table does not exist (dead server) —
a descriptive payload instead of propagating undefined.
%% Single catalog with 130 data rows (see loaded_catalogs/0) + 1 header.
1> erli18n_server:memory_info().
#{ets_bytes => 24576, num_catalogs => 1, num_keys => 131}See also loaded_catalogs/0, which_keys/2.
-spec reload(domain(), locale(), file:filename()) -> ensure_result().
Atomic reload of a .po catalog. Same as reload/4 with #{}.
Unlike ensure_loaded/3, it NEVER takes the idempotent fast-path: it always
parses and reinstalls, overwriting the old catalog entry by entry (AMB-001). It
never returns {ok, already}. See reload/4 for the atomic STAGE -> SWAP
semantics and the zero-miss-window guarantee.
1> erli18n_server:reload(my_domain, <<"fr">>, "fr.po").
{ok, 128}
%% An invalid .po does NOT destroy the good catalog in use:
2> erli18n_server:reload(my_domain, <<"fr">>, "broken.po").
{error, {parse_error, ...}}
3> erli18n_server:lookup_singular(my_domain, <<"fr">>, undefined, <<"Hello">>).
{ok, <<"Bonjour">>}See reload/4, ensure_loaded/3.
-spec reload(domain(), locale(), file:filename(), opts()) -> ensure_result().
Atomic reload of a .po catalog with resource options (STAGE -> SWAP).
reload is an atomic STAGE -> SWAP. The entire failable half (read, parse, compile
plural, CLDR divergence) runs in the CALLING process into an in-memory staged/0
WITHOUT touching the ETS, so a reload whose new .po is invalid (syntax error,
unsupported charset, bad Plural-Forms, missing file) returns a structured
{error, _} and leaves the previous catalog FULLY INTACT — never destroyed. On
success, the only observable mutation is insert-before-prune: each retained key
is overwritten old->new by an atomic ets:insert/2, and only the keys absent
from the new catalog are pruned afterwards — a concurrent reader of a retained
key never observes a miss window.
The heavy phase runs inside the span [erli18n, catalog, reload]; only the swap
commit travels to the server, with a tunable timeout. Opts is identical to
ensure_loaded/4's (include_fuzzy, max_bytes, max_entries, timeout).
Returns {ok, NewlyLoaded} or {error, ensure_error()} (never {ok, already}).
Edge cases
- Transient memory doubles during the swap (old + new rows coexist) — a deliberate cost of atomicity.
- Stale keys (present in the old catalog, absent in the new) serve the OLD translation for microseconds before the prune — consistent with gettext, better than a miss.
- The same
bound_error()s asensure_loaded/4apply; on any error the previous catalog stays intact.
1> erli18n_server:reload(my_domain, <<"fr">>, "fr.po", #{timeout => 30000}).
{ok, 128}See reload/3, ensure_loaded/4, opts().
-spec start_link() -> gen_server:start_ret().
Starts the catalog gen_server, registered locally as erli18n_server.
Called by the supervisor — in general you do NOT call this by hand. In init/1
the server claims the data ETS table from erli18n_table_owner via
'ETS-TRANSFER' (becoming its owner/writer) and (re)builds the O(1) catalog
index from the surviving rows.
Failure modes
init/1 crashes with {ets_handoff_timeout, _} if the owner does not hand the
table over within 5s (something structurally broken in the supervision tree),
which makes the supervisor re-evaluate. Since the owner is the table's heir, a
crash of THIS worker does not destroy the catalogs: the table returns to the
owner and is re-handed on the next init/1.
1> {ok, Pid} = erli18n_server:start_link().
{ok, <0.123.0>}
2> is_pid(Pid).
trueSee also init/1.
No cleanup to do: the data table is protected and has a heir (the owner), so
on a crash of this worker it returns INTACT to the owner instead of being
destroyed — exactly the scenario this architecture protects. The secondary index
is server-owned and dies with the worker, being rebuilt on the next init/1.
Returns ok.
Removes the (Domain, Locale) catalog entirely (entries + header).
Return and effects
O(catalog size) deletion, not O(total rows): it reads the data keys from the
secondary index and deletes each one (O(1) per key on a set), then removes the
header by its own key and deregisters the catalog from the index (finding #13).
After the unload, lookup_* of that catalog starts returning undefined.
Synchronous; always returns ok.
Idempotent: unloading a never-loaded catalog is a no-op (also returns ok).
Emits the telemetry span [erli18n, catalog, unload] whose stop metadata
includes result (ok | not_loaded) and keys_removed.
Failure modes
A non-atom Domain / non-binary Locale crash with function_clause.
1> erli18n_server:ensure_loaded(my_domain, <<"fr">>, "fr.po").
{ok, 128}
2> erli18n_server:unload(my_domain, <<"fr">>).
ok
3> erli18n_server:lookup_header(my_domain, <<"fr">>).
undefined
4> erli18n_server:unload(my_domain, <<"fr">>). %% idempotent
okSee also reload/3 (which does NOT do delete-then-reload), loaded_catalogs/0.
-spec which_keys(domain(), locale()) -> [{singular, context(), msgid()} | {plural, context(), msgid()}].
Enumerates the keys (singular and plural) loaded for (Domain, Locale).
Returns a SORTED list of {singular, Context, Msgid} and
{plural, Context, Msgid}. Plural entries are DEDUPLICATED by
(Context, Msgid): a plural msgid with N forms appears ONCE, not N times (the
loaded_catalogs/0 count, by contrast, counts each form). Parity with
gettexter:which_keys/2.
ETS scan (ets:tab2list/1) in the calling process, with no roundtrip to the
gen_server — observability, not a hot path. Absent catalog -> empty list.
1> erli18n_server:insert_singular(d, <<"fr">>, undefined, <<"Hello">>, <<"Bonjour">>).
ok
2> erli18n_server:insert_plural(d, <<"fr">>, undefined, <<"file">>,
.. [{0, <<"fichier">>}, {1, <<"fichiers">>}]).
ok
3> erli18n_server:which_keys(d, <<"fr">>).
[{plural, undefined, <<"file">>}, {singular, undefined, <<"Hello">>}]See also loaded_catalogs/0, memory_info/0.