KiwiCodec.Schema.Tokenizer (kiwi_codec v0.2.0)

Copy Markdown View Source

Tokenizes .kiwi schema text with source positions.

Summary

Functions

tokenize(text)

Functions

tokenize(text)

@spec tokenize(String.t()) :: [KiwiCodec.Schema.Token.t()]