Penelope v0.1.1 Penelope.NLP.Tokenizer behaviour

The behaviour implemented by all tokenizers.

Link to this section Summary

Link to this section Callbacks

Link to this callback tokenize(arg0)
tokenize(String.t()) :: [String.t()]