Language stopword lists used by SearchCore.Pipeline to drop high-frequency,
low-signal words before stemming.
Lists are stored as MapSets of lowercased, accented surface forms (matching is
done before accent folding). Languages without a bundled list return an empty set,
so the pipeline simply keeps every token for them.
Lookups are keyed by SearchCore.Language.base/1, so an algorithm variant gets its
language's list.
Summary
Functions
Languages that ship a bundled stopword list, as canonical ISO codes.
Functions
@spec available_languages() :: [SearchCore.Language.t()]
Languages that ship a bundled stopword list, as canonical ISO codes.
@spec get(SearchCore.Language.t()) :: MapSet.t(String.t())
Return the stopword MapSet for lang, or an empty set if none is bundled.
Keyed by SearchCore.Language.base/1, so an algorithm variant gets its language's
list: :en_porter is still English.
iex> SearchCore.Stopwords.get(:en_porter) == SearchCore.Stopwords.get(:en)
true