View Source Grammar.Tokenizer.TokenExtractorHelper (Grammar v0.1.0)

This module provides helper functions to work with TokenExtractor implementations using Regex.

Summary

Functions

Normalizes a regex to ensure it starts with ^, while keeping the original options.

Tries to read a token from the input string using a regex pattern.

Functions

@spec normalize_regex(Regex.t()) :: Regex.t()

Normalizes a regex to ensure it starts with ^, while keeping the original options.

Link to this function

try_read_from_regex(pattern, input_string)

View Source
@spec try_read_from_regex(Regex.t(), String.t()) :: nil | {String.t(), integer()}

Tries to read a token from the input string using a regex pattern.