Simple Bayes v0.7.1 SimpleBayes.Trainer.TokenParser

Summary

Functions

Parses a training string into tokens

Functions

parse(data, string, opts)

Parses a training string into tokens.

Examples

iex> SimpleBayes.Trainer.TokenParser.parse(
iex>   %SimpleBayes{},
iex>   "cute dog",
iex>   [default_weight: 1, stop_words: []]
iex> )
{
  ["cute", "dog"],
  %SimpleBayes{tokens: %{"cute" => 1, "dog" => 1}}
}

iex> SimpleBayes.Trainer.TokenParser.parse(
iex>   %SimpleBayes{tokens:  %{"cute" => 1, "cat" => 1}},
iex>   "cute dog",
iex>   [default_weight: 1, stop_words: []]
iex> )
{
  ["cute", "dog"],
  %SimpleBayes{tokens: %{"cute" => 2, "cat" => 1, "dog" => 1}}
}