Stemmers (stemmers v0.1.0)

Copy Markdown View Source

Snowball stemming for Elixir, backed by the Rust rust-stemmers crate through a Rustler NIF.

Unlike the pure-Elixir stemmer package (English only), this supports the full Snowball language set — French included — and is a maintained replacement for the abandoned stemex.

iex> Stemmers.stem("mangeant", :french)
"mang"

iex> Stemmers.stem("running", :english)
"run"

Safety

rust-stemmers is safe Rust (no unsafe), and Rustler converts any Rust panic into an Elixir exception, so a bad input cannot bring down the BEAM. stem_all/2 is scheduled on a dirty CPU scheduler so large batches never stall a normal scheduler thread.

Summary

Types

language()

Functions

stem(word, lang)

Stem a single word in the given language.

stem_all(words, lang)

Stem a list of words in the given language, reusing one stemmer instance.

supported?(lang)

Whether lang is a supported Snowball language.

supported_languages()

The list of supported Snowball languages, as atoms.

Types

language()

@type language() ::
  :arabic
  | :danish
  | :dutch
  | :english
  | :finnish
  | :french
  | :german
  | :greek
  | :hungarian
  | :italian
  | :norwegian
  | :portuguese
  | :romanian
  | :russian
  | :spanish
  | :swedish
  | :tamil
  | :turkish

Functions

stem(word, lang)

@spec stem(String.t(), language()) :: String.t()

Stem a single word in the given language.

Raises ArgumentError on an unsupported language so callers get a clear message instead of an opaque NIF decode error.

stem_all(words, lang)

@spec stem_all([String.t()], language()) :: [String.t()]

Stem a list of words in the given language, reusing one stemmer instance.

supported?(lang)

@spec supported?(atom()) :: boolean()

Whether lang is a supported Snowball language.

supported_languages()

@spec supported_languages() :: [language()]

The list of supported Snowball languages, as atoms.