Essence.Tokenizer.tokenize_s
You're seeing just the function
tokenize_s
, go back to Essence.Tokenizer module for more information.
Link to this function
tokenize_s(stream)
Specs
tokenize_s(File.Stream.t()) :: List.t()
Tokenizes a given stream
, and returns a list of tokens
. Commonly the given stream
is a :line
stream.