The characters a token's match can begin with, worked out from its IR when a parser is generated.

A generated tokenizer does maximal munch: at every position it tries every token and keeps the longest match. Most of those tries fail on the very first character, so the generated lex_candidates/2 asks each token's start set first (Grammar.Native.TokenizerCompiler) and tries only the tokens that can begin with the next character.

Too much, never too little. Where a start set cannot be told exactly -- any character (.), a @native(...) token, a token that reaches itself again before consuming anything -- the answer is :any, and the token is always tried. A start set that holds a character the token cannot really begin with only costs a try; one that missed a character would change which token wins. Only the characters a non-empty match can begin with count: a zero-width match never wins maximal munch.