LocaleSlug (LocaleSlug v0.2.0)

Copy Markdown View Source

Locale-aware and script-aware URL slugs.

LocaleSlug.slugify("Größe Fußball", locale: "de")   #=> "groesse-fussball"
LocaleSlug.slugify("Töö õun", locale: "et")         #=> "too-oun"
LocaleSlug.slugify("Цветокоррекция")                #=> "tsvetokorrektsiya"
LocaleSlug.slugify("Καλημέρα")                      #=> "kalimera"
LocaleSlug.slugify("日本語 入門", script: :native)    #=> "日本語-入門"

Why this exists

Slug generation is locale-dependent and no other Elixir package treats it that way. ö must become oe in German and o in Estonian — slugger gets German right and Estonian wrong, slugify the reverse, and the maintained transliterators (any_ascii, unidecode) expose arity-1 functions that cannot take a locale. The only locale-aware machinery in Elixir is an ICU C NIF, unusable in a library.

This package is pure Elixir with no dependencies.

Two axes

:locale chooses locale preferences (German ö→oe) and, for languages with their own official romanization, the schema (uk → Ukraine's KMU 55:2010).

:script chooses the output alphabet. :latin romanizes; :native keeps the source script and only makes it URL-safe, which is legal in URLs (RFC 3987) and is what Russian and Japanese sites routinely do.

What it will not do

Lossy romanization merges words, and that is not fixable in ASCII. Turkish ılık and ilik both slug to ilik — six of Turkish's letters fold onto bare ASCII, and the fold is the Turkish convention. Slugs are not identifiers; uniqueness belongs to your schema. Use script: :native when you need the distinction preserved.

Kanji romanization is out of scope: the reading is a property of the word, not the character (生物 is seibutsu or namamono), and it is an open research problem. script: :native handles Japanese correctly today.

Summary

Types

What to do with a script that has no romanizer under :latin.

A BCP 47 language tag. nil means neutral — no locale preferences.

L1 scheme selection. A bare atom or string applies to whichever script defines it; a map addresses several scripts at once, e.g. %{cyrillic: :kmu_2010}.

Output alphabet. :latin romanizes; :native keeps the source script and only makes it URL-safe. An open enum so a third script is additive rather than breaking.

Provenance for one table. :status is the trust level — prefer "verified" for slugs you persist. :deviations records where a table knowingly departs from the standard it cites, rather than hiding it.

Functions

Metadata for a locale or schema: :status, :source, :reviewers, :deviations.

Locale tags that ship an L2 table.

Resolves a locale tag without slugifying, so a caller can own locale-miss logging.

L1 schema ids that ship.

Turns text into a slug.

Romanizes text without applying any slug formatting.

Types

fallback()

@type fallback() :: :native | :empty

What to do with a script that has no romanizer under :latin.

locale()

@type locale() :: String.t() | atom() | nil

A BCP 47 language tag. nil means neutral — no locale preferences.

option()

@type option() ::
  {:locale, locale()}
  | {:script, script()}
  | {:separator, String.t()}
  | {:max_length, pos_integer() | nil}
  | {:schema, schema()}
  | {:fallback, fallback()}
  | {:strict, boolean()}

options()

@type options() :: [option()]

schema()

@type schema() :: atom() | String.t() | %{optional(atom()) => atom() | String.t()}

L1 scheme selection. A bare atom or string applies to whichever script defines it; a map addresses several scripts at once, e.g. %{cyrillic: :kmu_2010}.

Strings are accepted wherever an atom is, so a value read from config or an env var can be passed straight through without String.to_atom/1.

script()

@type script() :: :latin | :native

Output alphabet. :latin romanizes; :native keeps the source script and only makes it URL-safe. An open enum so a third script is additive rather than breaking.

table_info()

@type table_info() :: %{
  id: String.t(),
  kind: :base | :script | :locale,
  script: :cyrillic | :greek | :base | nil,
  status: String.t(),
  source: String.t(),
  reviewers: [String.t()],
  deviations: [String.t()]
}

Provenance for one table. :status is the trust level — prefer "verified" for slugs you persist. :deviations records where a table knowingly departs from the standard it cites, rather than hiding it.

Functions

info(id)

@spec info(String.t()) :: table_info() | nil

Metadata for a locale or schema: :status, :source, :reviewers, :deviations.

Prefer status: "verified" tables for slugs you persist. A clean machine audit is never verified — that label means a human who reads the language signed off.

locales()

@spec locales() :: [String.t()]

Locale tags that ship an L2 table.

resolve(tag)

Resolves a locale tag without slugifying, so a caller can own locale-miss logging.

{:ok, %LocaleSlug.Resolver{}}
{:missing, normalized_tag, %LocaleSlug.Resolver{}}
{:invalid, :country_only | :not_a_language_tag}

slugify/2 never writes to Logger — a pure string function called on every page render must not — so this is how you find out.

schemas()

@spec schemas() :: [String.t()]

L1 schema ids that ship.

slugify(text, opts \\ [])

@spec slugify(String.t() | any(), options()) :: String.t()

Turns text into a slug.

Options

  • :locale — BCP 47 tag ("de", "et-EE", :et). Default nil = neutral.
  • :script:latin (default) or :native.
  • :separator — default "-".
  • :max_length — in output characters. Never splits a rule's output, so щ→shch is kept whole or dropped entirely.
  • :schema — override the L1 scheme for a script. An atom, or a map like %{cyrillic: :kmu_2010}.
  • :fallback:native (default) or :empty, for scripts with no romanizer.
  • :strict — raise on a malformed :locale rather than falling back to neutral.

An unknown :schema raises with or without :strict, because unlike a locale it is never derived from request data — it is named at the call site, so an id with no table is a typo rather than a language we do not cover. Neither option ever raises on unromanizable text.

"" is a meaningful result — "no URL segment" — returned for input that is entirely punctuation, emoji or whitespace. The library never invents filler.

transliterate(text, opts \\ [])

@spec transliterate(String.t() | any(), options()) :: String.t()

Romanizes text without applying any slug formatting.

Letters are mapped exactly as slugify/2 maps them — same layers, same contextual rules — but spacing, punctuation and case-folding-only characters are left alone, and nothing is collapsed, trimmed or truncated.

LocaleSlug.transliterate("Цветокоррекция")  #=> "tsvetokorrektsiya"
LocaleSlug.transliterate("ülo.kask")        #=> "ulo.kask"
LocaleSlug.transliterate("日本")             #=> "日本"

Use this when the result feeds something other than a URL segment — a username, a filename, a sort key — and the caller has its own formatting rules. Anything with no romanizer passes through unchanged, so callers must still handle non-ASCII if they require ASCII.

Accepts the same :locale and :schema options as slugify/2.