Available Transforms

This document lists all transforms available in unicode_transform, including the :from and :to option atoms, direct transform IDs for the :transform option, and the 380 CLDR XML rule sets that ship with unicode_transform.

Using transforms

There are three ways to invoke a transform:

# 1. :from / :to atoms (most convenient)
iex> Unicode.Transform.transform("Москва", from: :cyrillic, to: :latin)
{:ok, "Moskva"}

# 2. :transform string (for transforms without atom mappings)
iex> Unicode.Transform.transform("Հayaստan", transform: "Armenian-Latin-BGN")
{:ok, "Hayastan"}

# 3. :transform with :direction (for reverse transforms)
iex> Unicode.Transform.transform("アイウエオ", transform: "Hiragana-Katakana", direction: :reverse)
{:ok, "あいうえお"}

The :from option defaults to :any when omitted:

# These are equivalent:
iex> Unicode.Transform.transform("hello", to: :upper)
{:ok, "HELLO"}

iex> Unicode.Transform.transform("hello", from: :any, to: :upper)
{:ok, "HELLO"}

Both atoms and strings are accepted for :from and :to. The BCP47 script code or the Unicode script name are both acceptable.

iex> Unicode.Transform.transform("αβγδ", from: :greek, to: :latin)
{:ok, "abgd"}

iex> Unicode.Transform.transform("αβγδ", from: "Greek", to: "Latin")
{:ok, "abgd"}

iex> Unicode.Transform.transform("αβγδ", from: :grek, to: :latn)
{:ok, "abgd"}

Automatic Script detection

Use from: :detect to automatically detect scripts in the input:

iex> Unicode.Transform.transform("αβγδ мир", from: :detect, to: :latin)
{:ok, "abgd mir"}

When from: :detect is set, the string is analzed to identify the script or scripts used within it. A transform is applied for each script identified in the string.

Programmatic listing

To list all available transforms at runtime:

iex> Unicode.Transform.available_transforms() |> Enum.take(5)
["Amharic-Latin-BGN", "Any-Accents", "Any-Publishing", "Arabic-Latin-BGN", "Arabic-Latin"]

This returns a sorted list of all transform IDs unicode_transform can resolve.

Built-in transforms

These transforms are implemented directly in Elixir without CLDR XML rule sets.

Script name atoms

These atoms can be used with the :from and :to options. Each script has a full name atom and a BCP47 (ISO 15924) code atom that resolve to the same value.

Additional atoms without BCP47 equivalents:

Target atoms

Special atoms

Script-to-Latin transforms

These are the most commonly used transforms. Use from: <script>, to: :latin or the equivalent :transform string.

Bidirectional transforms can be used in reverse with from: :latin, to: <script>.

Latin-ASCII (accent stripping)

iex> Unicode.Transform.transform("café résumé", from: :latin, to: :ascii)
{:ok, "cafe resume"}

This transform has a dedicated fast-path module for maximum performance.

Width transforms

iex> Unicode.Transform.transform("Ｈｅｌｌｏ", transform: "Fullwidth-Halfwidth")
{:ok, "Hello"}

Japanese script conversion

iex> Unicode.Transform.transform("あいうえお", from: :hiragana, to: :katakana)
{:ok, "アイウエオ"}

Han (Chinese) transforms

iex> Unicode.Transform.transform("中国", transform: "Han-Latin")
{:ok, "zhōng guó"}

Indic cross-script transforms

Every pair of Indic scripts has a direct transform via the InterIndic pivot. These are all bidirectional.

Available Indic scripts

Bengali, Devanagari, Gujarati, Gurmukhi, Kannada, Malayalam, Oriya, Tamil, Telugu

Cross-script matrix

Each Indic script can be transformed to every other Indic script. For example:

iex> Unicode.Transform.transform("हिन्दी", from: :devanagari, to: :bengali)
{:ok, "হিন্দী"}

iex> Unicode.Transform.transform("বাংলা", from: :bengali, to: :gujarati)
{:ok, "બાંલા"}

The full set of Indic cross-script files:

Indic-to-Arabic and Indic-to-Urdu

Each Indic script also has transforms to Arabic script and Urdu:

InterIndic pivot transforms

These are internal transforms used by the cross-script Indic pipeline. They can also be used directly:

BGN/PCGN romanization transforms

These follow the BGN/PCGN romanization standards used by the U.S. Board on Geographic Names and the Permanent Committee on Geographical Names.

iex> Unicode.Transform.transform("Москва", transform: "Russian-Latin-BGN")
{:ok, "Moskva"}

UNGEGN romanization

Uses the United Nations Group of Experts on Geographical Names(United Nations Group of Experts on Geographical Names)[https://en.wikipedia.org/wiki/United_Nations_Group_of_Experts_on_Geographical_Names] Romanization rules.

Locale-specific transforms

German ASCII

iex> Unicode.Transform.transform("Ä ö ü", transform: "de-ASCII")
{:ok, "AE oe ue"}

# Also available via :from/:to
iex> Unicode.Transform.transform("Ä ö ü", from: "de", to: "ASCII")
{:ok, "AE oe ue"}

Locale-specific case transforms

These apply language-specific casing rules (e.g., Turkish dotted/dotless I, Lithuanian accent handling).

iex> Unicode.Transform.transform("İSTANBUL", transform: "tr-Lower")
{:ok, "istanbul"}

Ethiopic transforms

CLDR names these rule files by their BCP-47 identifier. Each row lists the forward alias followed by its BCP-47 form; transforms marked "both" also resolve in the reverse direction (for example Latin-Ethiopic/SERA or und-Ethi-t-und-latn-m0-sera).

Phonetic (IPA/FONIPA) transforms

Transforms to and from the [International Phonetic Alphabet](https://en.wikipedia.org/wiki/International_Phonetic_Alphabet#:~:text=The%20International%20Phonetic%20Alphabet%20(IPA,for%20the%20sounds%20of%20speech.).

IPA-based cross-language transforms

These use IPA as a pivot to approximate transliteration between languages.

Cross-language transliteration transforms

These transform between specific language pairs, often using phonetic approximation.

Specialized transforms

Other transforms

Myanmar/Zawgyi

Script variant transforms