SearchCore.Language (search_core v0.3.0)

Copy Markdown View Source

The languages the search stack accepts: ISO 639-1 codes, exactly the set the installed text_stemmer reports — :fr, :en, :de, …

iex> SearchCore.Language.supported?(:fr)
true
iex> SearchCore.Language.supported?(:klingon)
false

There is one spelling for a language and no aliases: search_core does not decide what a language is called, and does not carry a table that could disagree with the stemmer. Which languages exist is asked of Text.Stemmer at call time, never frozen in, so upgrading text_stemmer makes any language it adds usable immediately — no search_core release, no recompile.

Need a human-readable, localized language name for a UI? That is what the ISO code is for: hand it to ex_cldr_languages, which does it properly in every locale. Storing English names here would be a worse copy of a solved problem.

Algorithm variants

Snowball ships more than one algorithm for a few languages, addressed by a qualified code: :en_porter, :en_lovins, :nl_porter. A variant still belongs to its language — :en_porter is English, so it takes the English stopword list — which is what base/1 is for. Use base/1, not the raw code, for anything keyed by language rather than by algorithm.

Summary

Functions

The language an algorithm variant belongs to: :en_porter:en. Any other supported code is returned unchanged; nil when unsupported.

Whether the installed stemmer supports lang.

Every language the installed stemmer supports, including algorithm variants.

Return lang, raising ArgumentError unless the installed stemmer supports it.

Types

t()

@type t() :: atom()

Functions

base(lang)

@spec base(any()) :: t() | nil

The language an algorithm variant belongs to: :en_porter:en. Any other supported code is returned unchanged; nil when unsupported.

iex> SearchCore.Language.base(:en_porter)
:en
iex> SearchCore.Language.base(:fr)
:fr

supported?(lang)

@spec supported?(any()) :: boolean()

Whether the installed stemmer supports lang.

supported_languages()

@spec supported_languages() :: [t()]

Every language the installed stemmer supports, including algorithm variants.

This is the set to validate user-facing input against, and what one_of: constraints should use.

validate!(lang)

@spec validate!(any()) :: t()

Return lang, raising ArgumentError unless the installed stemmer supports it.

iex> SearchCore.Language.validate!(:fr)
:fr