Simple Bayes v0.7.1 SimpleBayes.Trainer.TokenParser
Summary
Functions
Parses a training string into tokens
Functions
Parses a training string into tokens.
Examples
iex> SimpleBayes.Trainer.TokenParser.parse(
iex> %SimpleBayes{},
iex> "cute dog",
iex> [default_weight: 1, stop_words: []]
iex> )
{
["cute", "dog"],
%SimpleBayes{tokens: %{"cute" => 1, "dog" => 1}}
}
iex> SimpleBayes.Trainer.TokenParser.parse(
iex> %SimpleBayes{tokens: %{"cute" => 1, "cat" => 1}},
iex> "cute dog",
iex> [default_weight: 1, stop_words: []]
iex> )
{
["cute", "dog"],
%SimpleBayes{tokens: %{"cute" => 2, "cat" => 1, "dog" => 1}}
}