defmodule BeamlabLanguages do
@moduledoc """
Linguistic metadata for human languages.
Answers questions like:
- Does this language use grammatical gender? Which genders?
- Is it written right-to-left?
- What's the canonical English name? The endonym?
- Can I collapse a BCP 47 tag like `"en-US"` to a base code?
All data is curated and embedded at compile time. No runtime file I/O,
no GenServer, no ETS, no runtime dependencies.
## Gender codes
Genders are returned as strings. Consumers commonly see `"m"` (masculine),
`"f"` (feminine), and `"n"` (neuter), but **also `"c"` (common)** for the
Continental Scandinavian and Dutch systems where masculine and feminine
have merged: Danish, Dutch, Norwegian Bokmål via `no`, and Swedish all
use `["c", "n"]`. Pattern-match on all four — a `case g do "m" -> ...;
"f" -> ...; "n" -> ... end` will silently miss those languages.
## Verb conjugation
`has_verb_conjugation?/1`, `verb_groups/1`, `persons/1`, `persons/2`,
`conjugation_paradigm/1`, `tense_level/3`, and `reflexive?/2` expose
pedagogical conjugation metadata for language-learning UIs: the
modes/tenses a learner is taught, the proficiency level each tense is
taught at, the group system (e.g. French -er/-ir/-re), the pronoun list
(each tagged with grammatical `:number`), and whether a given lemma is a
reflexive / pronominal verb (`reflexive?("fr", "se laver")`,
`reflexive?("it", "chiamarsi")`).
The contract is **"true iff we've curated a paradigm"**, not "true iff
the language inflects verbs". So `has_verb_conjugation?("fr")` is `true`,
`has_verb_conjugation?("zh")` is `false`, and `has_verb_conjugation?("en")`
is also `false` until an English paradigm is curated. v0.2 ships French
only — more languages will be added as consumers need them.
Every label entry carries both `:label_native` (the term in the target
language, e.g. `"Indicatif"`) and `:label_en` (the canonical English
rendering, e.g. `"Indicative"`). Tenses additionally carry a `:level`
(CEFR / JLPT / HSK key) marking where they sit in the curriculum. Order
in every list is the **teaching order** — opinionated and stable across
versions.
## Quick start
iex> BeamlabLanguages.has_gender?("fr")
true
iex> BeamlabLanguages.genders("de")
["m", "f", "n"]
iex> BeamlabLanguages.direction("ar")
:rtl
iex> BeamlabLanguages.normalize("en-US")
"en"
iex> BeamlabLanguages.has_verb_conjugation?("fr")
true
Every function that takes a language code runs `normalize/1` on it
internally — pass `"en-US"`, `"FR"`, or `" fr "` and lookups still work.
## Proficiency levels
`level_systems/0`, `levels/1`, `level_system_label/1`, and
`level_info/2` expose curated proficiency level systems (CEFR,
JLPT, HSK) for language-learning UIs. Order is pedagogical
(A1→C2, N5→N1, HSK1→HSK6), not alphabetical.
To go the other way — from a language to its system — use
`level_system/1` (`"fr"` → `"cefr"`, `"zh"` → `"hsk"`, `"ja"` →
`"jlpt"`) and `language_levels/1` (the level keys for a language in
one call). CEFR is the default for any language without a more
specific system; Korean returns `nil` (TOPIK isn't modeled).
## Roadmap
Planned for future versions and intentionally **not** in v1:
localized language names, plural rules, articles, case marking,
noun classes, scripts, IPA inventory, honorific levels. Verb
conjugation paradigms ship per-language as consumers need them
(French only as of v0.2).
"""
alias BeamlabLanguages.Conjugation
alias BeamlabLanguages.Language
alias BeamlabLanguages.Levels
alias BeamlabLanguages.Reflexive
@type code :: String.t()
@type gender :: String.t()
@type direction :: :ltr | :rtl
@data_path Path.join([:code.priv_dir(:beamlab_languages), "data", "languages.json"])
@external_resource @data_path
@raw @data_path
|> File.read!()
|> JSON.decode!()
# Map deprecated / regional bases to the canonical entry in the data file.
# Real-world input from POSIX locales, glibc, and browsers emits these:
# - "nb" (Bokmål) and "nn" (Nynorsk) collapse to "no" (Norwegian)
@aliases %{"nb" => "no", "nn" => "no"}
# Maps a language base code to its proficiency level system. CEFR is the
# default for any known language without a more specific system, so only
# the exceptions are listed here:
# - "zh" (Chinese, and its variants via normalize/1) => "hsk"
# - "ja" (Japanese) => "jlpt"
# - "ko" (Korean) => nil — TOPIK is the real-world system but is not
# among the three we curate. Defaulting it to CEFR would be wrong, so
# it's an explicit nil: a known gap, not an oversight.
@level_systems %{"zh" => "hsk", "ja" => "jlpt", "ko" => nil}
@languages @raw
|> Enum.map(fn {code, data} ->
{code,
%Language{
code: code,
name: data["name"],
native_name: data["native_name"],
direction: String.to_atom(data["direction"]),
has_gender: data["has_gender"],
genders: data["genders"]
}}
end)
|> Map.new()
@sorted_codes @languages |> Map.keys() |> Enum.sort()
@sorted_languages Enum.map(@sorted_codes, &Map.fetch!(@languages, &1))
@doc """
Returns the language struct for a code, or `nil` if unknown.
Accepts BCP 47 input — `"en-US"`, `"zh-Hans-CN"` — and sloppy casing.
Lookups are normalized internally via `normalize/1`.
## Examples
iex> BeamlabLanguages.get("fr").name
"French"
iex> BeamlabLanguages.get("en-US").code
"en"
iex> BeamlabLanguages.get("xx")
nil
"""
@spec get(any()) :: Language.t() | nil
def get(code) do
case normalize(code) do
nil -> nil
base -> Map.get(@languages, base)
end
end
@doc """
Lists every known language struct, sorted by code.
Sort order is stable so the result can drive UI dropdowns without flicker.
## Examples
iex> langs = BeamlabLanguages.list()
iex> hd(langs).__struct__
BeamlabLanguages.Language
iex> length(langs) > 0
true
"""
@spec list() :: [Language.t()]
def list, do: @sorted_languages
@doc """
Lists every known 2-letter base code, sorted.
## Examples
iex> "en" in BeamlabLanguages.list_codes()
true
iex> codes = BeamlabLanguages.list_codes()
iex> codes == Enum.sort(codes)
true
"""
@spec list_codes() :: [code()]
def list_codes, do: @sorted_codes
@doc """
Returns true iff the language uses grammatical gender.
Returns `false` for unknown / `nil` / non-string input rather than
raising — callers (form validation, template rendering) often pass
whatever they received from the user.
## Examples
iex> BeamlabLanguages.has_gender?("fr")
true
iex> BeamlabLanguages.has_gender?("en")
false
iex> BeamlabLanguages.has_gender?("xx")
false
"""
@spec has_gender?(any()) :: boolean()
def has_gender?(code) do
case get(code) do
%Language{has_gender: g} -> g
nil -> false
end
end
@doc """
Returns the list of gender codes a language uses.
Returns `[]` for languages without grammatical gender, and `[]` for
unknown codes.
## Examples
iex> BeamlabLanguages.genders("fr")
["m", "f"]
iex> BeamlabLanguages.genders("de")
["m", "f", "n"]
iex> BeamlabLanguages.genders("en")
[]
"""
@spec genders(any()) :: [gender()]
def genders(code) do
case get(code) do
%Language{genders: g} -> g
nil -> []
end
end
@doc """
Returns the writing direction.
Returns `:ltr` for unknown codes — the safe default for most rendering
contexts where an unknown language shouldn't flip the page layout.
## Examples
iex> BeamlabLanguages.direction("ar")
:rtl
iex> BeamlabLanguages.direction("en")
:ltr
iex> BeamlabLanguages.direction("xx")
:ltr
"""
@spec direction(any()) :: direction()
def direction(code) do
case get(code) do
%Language{direction: d} -> d
nil -> :ltr
end
end
@doc """
Canonical English name of the language. Returns `nil` for unknown codes.
## Examples
iex> BeamlabLanguages.name("fr")
"French"
iex> BeamlabLanguages.name("ja")
"Japanese"
iex> BeamlabLanguages.name("xx")
nil
"""
@spec name(any()) :: String.t() | nil
def name(code) do
case get(code) do
%Language{name: n} -> n
nil -> nil
end
end
@doc """
Native (endonym) name of the language — what speakers call it themselves.
Returns `nil` for unknown codes.
## Examples
iex> BeamlabLanguages.native_name("fr")
"Français"
iex> BeamlabLanguages.native_name("ja")
"日本語"
iex> BeamlabLanguages.native_name("xx")
nil
"""
@spec native_name(any()) :: String.t() | nil
def native_name(code) do
case get(code) do
%Language{native_name: n} -> n
nil -> nil
end
end
@doc """
Normalizes a language input string to a 2-letter base code.
- Strips dialect tags (`"en-US"` → `"en"`, `"zh-Hans-CN"` → `"zh"`)
- Accepts `_` as a separator too (`"en_US"` → `"en"`)
- Lowercases (`"FR"` → `"fr"`)
- Trims whitespace
- Maps deprecated / regional bases to their canonical entry: `"nb"`
(Bokmål) and `"nn"` (Nynorsk) collapse to `"no"` (Norwegian)
- Returns `nil` if no plausible 2-letter base can be extracted
This is what every other function calls internally before looking up
a code, so consumers never need to normalize before calling `get/1`,
`name/1`, etc. — but it's exposed because consumers sometimes need
the bare base code for their own purposes.
## Examples
iex> BeamlabLanguages.normalize("en-US")
"en"
iex> BeamlabLanguages.normalize("FR")
"fr"
iex> BeamlabLanguages.normalize("zh-Hans-CN")
"zh"
iex> BeamlabLanguages.normalize("nb-NO")
"no"
iex> BeamlabLanguages.normalize("")
nil
iex> BeamlabLanguages.normalize(nil)
nil
"""
@spec normalize(any()) :: code() | nil
def normalize(input) when is_binary(input) do
input
|> String.trim()
|> String.downcase()
|> String.split(["-", "_"], parts: 2)
|> hd()
|> base_or_nil()
|> dealias()
end
def normalize(_), do: nil
defp base_or_nil(<>) when a in ?a..?z and b in ?a..?z, do: <>
defp base_or_nil(_), do: nil
defp dealias(nil), do: nil
defp dealias(base), do: Map.get(@aliases, base, base)
@doc """
Returns true iff the code maps to a known language.
Sugar over `get/1`. Returns `false` for unknown / `nil` / non-string input.
## Examples
iex> BeamlabLanguages.known?("fr")
true
iex> BeamlabLanguages.known?("en-US")
true
iex> BeamlabLanguages.known?("xx")
false
"""
@spec known?(any()) :: boolean()
def known?(code), do: get(code) != nil
@doc """
Returns true iff a verb conjugation paradigm is curated for the language.
The contract is data-driven: returns `true` exactly when
`conjugation_paradigm/1` would return non-`nil` for the same code.
English and Swedish technically inflect verbs but currently return
`false` — they have no curated paradigm yet.
Returns `false` for unknown / `nil` / non-string input.
## Examples
iex> BeamlabLanguages.has_verb_conjugation?("fr")
true
iex> BeamlabLanguages.has_verb_conjugation?("zh")
false
iex> BeamlabLanguages.has_verb_conjugation?("xx")
false
"""
@spec has_verb_conjugation?(any()) :: boolean()
def has_verb_conjugation?(code) do
case normalize(code) do
nil -> false
base -> Conjugation.has_paradigm?(base)
end
end
@doc """
Returns the pedagogical verb groups for a language, or `nil`.
Verb groups are the curriculum buckets used to teach conjugation
(French's -er / -ir / -re, Spanish's -ar / -er / -ir, etc.). Each
entry is a map with `:key`, `:label_native` (target language), and
`:label_en` (English).
Returns `nil` when no paradigm is curated, **and also** when the
language has a paradigm but no meaningful pedagogical group system.
## Examples
iex> groups = BeamlabLanguages.verb_groups("fr")
iex> length(groups)
3
iex> hd(groups)
%{key: "1", label_native: "1er groupe (verbes en -er)", label_en: "1st group (-er verbs)"}
iex> BeamlabLanguages.verb_groups("zh")
nil
iex> BeamlabLanguages.verb_groups("xx")
nil
"""
@spec verb_groups(any()) :: [map()] | nil
def verb_groups(code) do
case normalize(code) do
nil -> nil
base -> Conjugation.verb_groups(base)
end
end
@doc """
Returns the person/pronoun list for a language's conjugation, or `nil`.
Each entry is a map with `:key` (a stable identifier like `"1sg"` or
`"3pl"`), `:label_native` (the pronoun in the target language),
`:label_en` (the English gloss, useful for learner UIs), and `:number`
(`:singular`, `:plural`, or `:dual` for a future dual-marking language;
`nil` if the key carries no recognisable number). Order is the teaching
order — singular persons first, then plural — so the list can drive a
conjugation grid without a separate person-ordering table.
The set of person keys may vary by language — a future Slovenian entry
would add a dual, Arabic would split 2nd person by gender, etc. Don't
assume a fixed six-person shape; filter on `:number` (or use
`persons/2`) rather than slicing the list by position.
Returns `nil` for languages without a curated paradigm.
## Examples
iex> persons = BeamlabLanguages.persons("fr")
iex> length(persons)
6
iex> hd(persons)
%{key: "1sg", label_native: "je", label_en: "I", number: :singular}
iex> BeamlabLanguages.persons("zh")
nil
"""
@spec persons(any()) :: [map()] | nil
def persons(code) do
case normalize(code) do
nil -> nil
base -> Conjugation.persons(base)
end
end
@doc """
Like `persons/1`, but filters by grammatical number.
Pass `number: :singular`, `number: :plural`, or `number: :dual` to get only
the persons in that number, in teaching order. With no `:number` option this
is identical to `persons/1`. Lets a UI render singular and plural blocks
without hardcoding which person keys belong to each.
Returns `nil` for languages without a curated paradigm (same as `persons/1`),
and `[]` when the language has persons but none in the requested number.
## Examples
iex> BeamlabLanguages.persons("fr", number: :singular) |> Enum.map(& &1.key)
["1sg", "2sg", "3sg"]
iex> BeamlabLanguages.persons("fr", number: :plural) |> Enum.map(& &1.key)
["1pl", "2pl", "3pl"]
iex> BeamlabLanguages.persons("fr", number: :dual)
[]
iex> BeamlabLanguages.persons("zh", number: :singular)
nil
"""
@spec persons(any(), keyword()) :: [map()] | nil
def persons(code, opts) when is_list(opts) do
with list when is_list(list) <- persons(code) do
case Keyword.get(opts, :number) do
nil -> list
number -> Enum.filter(list, &(&1.number == number))
end
end
end
@doc """
Returns true iff the lemma is a reflexive / pronominal verb in the language.
Recognises the language's reflexive marker on a dictionary-form lemma:
French's leading pronoun (`"se laver"`, `"s'appeler"`) and Italian's enclitic
`-rsi` ending (`"chiamarsi"`, `"mettersi"`). The lemma is lowercased and
trimmed internally, so values straight from user input or a database column
work as-is.
The contract is **"true iff we recognise a reflexive marker for a language we
have a rule for"**. Returns `false` for languages without a curated reflexive
rule, for non-reflexive lemmas, and for unknown / `nil` codes or non-string
lemmas — callers routinely pass whatever they have.
## Examples
iex> BeamlabLanguages.reflexive?("fr", "se laver")
true
iex> BeamlabLanguages.reflexive?("fr", "s'appeler")
true
iex> BeamlabLanguages.reflexive?("fr", "manger")
false
iex> BeamlabLanguages.reflexive?("fr", "semer")
false
iex> BeamlabLanguages.reflexive?("it", "chiamarsi")
true
iex> BeamlabLanguages.reflexive?("it", "parlare")
false
iex> BeamlabLanguages.reflexive?("en", "wash oneself")
false
iex> BeamlabLanguages.reflexive?("xx", "se laver")
false
"""
@spec reflexive?(any(), any()) :: boolean()
def reflexive?(code, lemma) do
case normalize(code) do
nil -> false
base -> Reflexive.reflexive?(base, lemma)
end
end
@doc """
Returns the conjugation paradigm — modes and their tenses — or `nil`.
Shape: `%{modes: [%{key, label_native, label_en, tenses: [%{key,
label_native, label_en, level}, ...]}, ...]}`. Order of modes and
tenses is the **teaching order** — opinionated and stable across
versions.
Each tense carries a `:level` — the proficiency level (a CEFR key like
`"B1"` for European languages, or the relevant JLPT / HSK key for
zh / ja) at which the tense/mood is typically taught. It's a property
of the tense in the language's curriculum, independent of any specific
verb. `nil` when the level is genuinely unknown; every French tense has
a value. Use `tense_level/3` to read one without walking the tree.
Persons live separately under `persons/1`, not inside the paradigm,
so the same paradigm can be paired with the language's pronoun list
in the consumer UI.
Returns `nil` for languages without a curated paradigm.
## Examples
iex> paradigm = BeamlabLanguages.conjugation_paradigm("fr")
iex> length(paradigm.modes)
4
iex> [first | _] = paradigm.modes
iex> first.key
"indicatif"
iex> first.label_native
"Indicatif"
iex> first.label_en
"Indicative"
iex> length(first.tenses)
8
iex> hd(first.tenses)
%{key: "present", label_native: "Présent", label_en: "Present", level: "A1"}
iex> BeamlabLanguages.conjugation_paradigm("zh")
nil
"""
@spec conjugation_paradigm(any()) :: map() | nil
def conjugation_paradigm(code) do
case normalize(code) do
nil -> nil
base -> Conjugation.paradigm(base)
end
end
@doc """
Returns the proficiency level for a single tense, or `nil`.
Convenience reader over `conjugation_paradigm/1` so consumers don't have
to walk the modes/tenses tree. The level is a CEFR key (`"A1"`…`"C2"`)
for European languages, or the relevant JLPT / HSK key for zh / ja.
Returns `nil` for unknown / `nil` codes, unknown mode or tense keys, and
for tenses whose level is genuinely unknown.
## Examples
iex> BeamlabLanguages.tense_level("fr", "subjonctif", "present")
"B1"
iex> BeamlabLanguages.tense_level("fr", "indicatif", "present")
"A1"
iex> BeamlabLanguages.tense_level("fr", "indicatif", "nonexistent")
nil
iex> BeamlabLanguages.tense_level("zh", "indicatif", "present")
nil
"""
@spec tense_level(any(), String.t(), String.t()) :: String.t() | nil
def tense_level(code, mode_key, tense_key) do
case normalize(code) do
nil -> nil
base -> Conjugation.tense_level(base, mode_key, tense_key)
end
end
@doc """
Lists every known proficiency level system key, sorted.
## Examples
iex> "cefr" in BeamlabLanguages.level_systems()
true
iex> BeamlabLanguages.level_systems() == Enum.sort(BeamlabLanguages.level_systems())
true
"""
@spec level_systems() :: [String.t()]
def level_systems, do: Levels.systems()
@doc """
Lists the levels for a proficiency system, in pedagogical order.
Returns `[]` for unknown systems.
## Examples
iex> BeamlabLanguages.levels("cefr")
["A1", "A2", "B1", "B2", "C1", "C2"]
iex> BeamlabLanguages.levels("jlpt")
["N5", "N4", "N3", "N2", "N1"]
iex> BeamlabLanguages.levels("unknown")
[]
"""
@spec levels(String.t()) :: [String.t()]
def levels(system), do: Levels.level_keys(system)
@doc """
Returns the human-readable label for a proficiency system.
Returns `nil` for unknown systems.
## Examples
iex> BeamlabLanguages.level_system_label("cefr")
"CEFR"
iex> BeamlabLanguages.level_system_label("hsk")
"HSK"
iex> BeamlabLanguages.level_system_label("unknown")
nil
"""
@spec level_system_label(String.t()) :: String.t() | nil
def level_system_label(system), do: Levels.label(system)
@doc """
Returns metadata for a single level within a system.
Returns `nil` for unknown systems or unknown levels.
## Examples
iex> BeamlabLanguages.level_info("cefr", "A1")
%{key: "A1", label: "A1", description: "Beginner"}
iex> BeamlabLanguages.level_info("cefr", "Z9")
nil
iex> BeamlabLanguages.level_info("unknown", "A1")
nil
"""
@spec level_info(String.t(), String.t()) :: map() | nil
def level_info(system, level_key), do: Levels.level_info(system, level_key)
@doc """
Returns the proficiency level system for a language, or `nil`.
CEFR is the default for any known language without a more specific
system; Chinese (`"zh"`) maps to HSK and Japanese (`"ja"`) to JLPT.
Korean (`"ko"`) has no curated system — TOPIK isn't among the three we
model — so it returns `nil` rather than a wrong default. The result is a
system key suitable for `levels/1`, `level_system_label/1`, etc.
Accepts BCP 47 input and sloppy casing like every other code-taking
function — lookups are normalized via `normalize/1`. Returns `nil` for
unknown / `nil` / non-string input.
## Examples
iex> BeamlabLanguages.level_system("fr")
"cefr"
iex> BeamlabLanguages.level_system("fr-FR")
"cefr"
iex> BeamlabLanguages.level_system("zh")
"hsk"
iex> BeamlabLanguages.level_system("ja")
"jlpt"
iex> BeamlabLanguages.level_system("ko")
nil
iex> BeamlabLanguages.level_system("xx")
nil
"""
@spec level_system(any()) :: String.t() | nil
def level_system(code) do
case get(code) do
nil -> nil
%Language{code: base} -> Map.get(@level_systems, base, "cefr")
end
end
@doc """
Returns the proficiency level keys for a language, in pedagogical order.
Convenience for `levels(level_system(code))`: resolves the language's
level system and lists its keys. Returns `[]` when the language has no
curated system (e.g. Korean) and for unknown / `nil` codes — the same
empty result `levels/1` gives for an unknown system.
## Examples
iex> BeamlabLanguages.language_levels("fr")
["A1", "A2", "B1", "B2", "C1", "C2"]
iex> BeamlabLanguages.language_levels("zh")
["HSK1", "HSK2", "HSK3", "HSK4", "HSK5", "HSK6"]
iex> BeamlabLanguages.language_levels("ja")
["N5", "N4", "N3", "N2", "N1"]
iex> BeamlabLanguages.language_levels("ko")
[]
iex> BeamlabLanguages.language_levels("xx")
[]
"""
@spec language_levels(any()) :: [String.t()]
def language_levels(code) do
case level_system(code) do
nil -> []
system -> levels(system)
end
end
end