Rule-level helpers shared by every Grammar.Native.RuleCompiler-generated
function -- the Parser stage of Aether's Reader/Tokenizer/Lexer/Parser
split. Mirrors Grammar.VM.TokenInterpreter's own semantics exactly
(same backtracking discipline, same no-progress guard on Star), just
expressed as plain function composition instead of a bytecode
interpreter loop -- Grammar.VM.Compiler's moduledoc describes the
same combinators this module implements.
Every combinator here operates on a {:ok, pos, ref_stack, captures} | :fail shape, where captures is a raw capture map in exactly the
shape Ichor.Actions already expects ({:token, name, text} /
{:rule, name, sub_captures} / {:text, text}) -- generated code
builds this map compositionally as it goes, rather than through a
single mutable per-rule frame the way Grammar.VM.TokenInterpreter
does, since plain Elixir recursion already gives every failed
alternative's partial captures nowhere to leak to (they're just a
discarded return value, never mutated shared state).
Summary
Functions
Positive lookahead: consumes nothing either way.
Calls fun exactly n times in sequence, merging captures, failing outward if any call fails.
Concatenates the raw text of every token in [start_pos, end_pos) -- the :text capture kind's raw value.
@indent(expr): the current token's column must exceed the enclosing reference column; pushes its own column as the new reference for expr, pops it back off afterward.
Matches one stream token named name at pos, advancing by one -- the
raw building block every bare token reference compiles to. capture
is the token's own override (see Grammar.VM.Token) -- nil unless a
Grammar.IR.CustomLexeme matched it with one attached.
Folds every key of new into acc, list-appending (or list-wrapping)
on repeat capture names -- the same rule a rule's own capture frame
follows in the VM (Grammar.VM.TokenInterpreter's own private
merge_capture helper).
Negative lookahead: consumes nothing either way.
Zero or one: on failure, a zero-width success with no captures.
One or more, greedy: one mandatory match, then star/6 for the rest.
Bounded repetition {n,m}: min mandatory calls, then up to max - min more (or unbounded, when max == :infinity).
@samecol(expr): the current token's column must exactly match the enclosing reference column.
Skips tokens named skip_name at pos (mirrors Grammar.VM's own
private skip_leading_trivia/3, for the same run_sequence case:
between two top-level matches there's no enclosing sequence for
Aether.Parser's skip-splicing to have spliced anything into, so a
leading @skip token here is genuinely unconsumed leftover from the
previous top-level match). skip_name is nil for a @noskip
grammar, in which case pos is returned unchanged.
Zero or more, greedy. Guards against a body that matches without
consuming anything (Grammar.VM.Compiler's own test_progress doc
explains why this can't be left to the static lint alone) -- a
zero-width success is still incorporated once, then the loop stops
rather than looping forever.
Ordered PEG choice: the first alternative that succeeds wins, tried in order.
A token stream matched only a prefix of the input -- the same error Grammar.VM's own match/2 reports.
Types
@type rule_fun() :: (tuple(), non_neg_integer(), [integer()], term() -> rule_result())
@type rule_result() :: {:ok, non_neg_integer(), [integer()], captures()} | :fail
Functions
@spec and_pred(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
Positive lookahead: consumes nothing either way.
@spec call_n_times( rule_fun(), non_neg_integer(), tuple(), non_neg_integer(), [integer()], term() ) :: rule_result()
Calls fun exactly n times in sequence, merging captures, failing outward if any call fails.
@spec concat_text(tuple(), non_neg_integer(), non_neg_integer()) :: String.t()
Concatenates the raw text of every token in [start_pos, end_pos) -- the :text capture kind's raw value.
@spec indent_enter(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
@indent(expr): the current token's column must exceed the enclosing reference column; pushes its own column as the new reference for expr, pops it back off afterward.
@spec match_token(tuple(), non_neg_integer(), atom()) :: {:ok, non_neg_integer(), String.t(), term()} | :fail
Matches one stream token named name at pos, advancing by one -- the
raw building block every bare token reference compiles to. capture
is the token's own override (see Grammar.VM.Token) -- nil unless a
Grammar.IR.CustomLexeme matched it with one attached.
Folds every key of new into acc, list-appending (or list-wrapping)
on repeat capture names -- the same rule a rule's own capture frame
follows in the VM (Grammar.VM.TokenInterpreter's own private
merge_capture helper).
Unlike that VM function, this one has to merge whole subtree
capture maps at once (a Seq/Star node's already-accumulated
captures), not a single new occurrence at a time -- so a value that's
already a list (built up by a nested Star/Plus/Rep sharing this
same capture name) has to be flattened into the result rather than
wrapped as one more element, or [a, b, c] merged against a sibling
d would wrongly become [d, [a, b, c]] instead of [d, a, b, c].
A raw capture value is always a {:token, ...} / {:rule, ...} /
{:text, ...} tuple, never a bare list on its own, so "the value is
a list" unambiguously means "already an accumulation," never "one
occurrence that happens to look like a list."
@spec not_pred(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
Negative lookahead: consumes nothing either way.
@spec opt(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
Zero or one: on failure, a zero-width success with no captures.
@spec plus(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
One or more, greedy: one mandatory match, then star/6 for the rest.
@spec rep( rule_fun(), non_neg_integer(), non_neg_integer() | :infinity, tuple(), non_neg_integer(), [integer()], term() ) :: rule_result()
Bounded repetition {n,m}: min mandatory calls, then up to max - min more (or unbounded, when max == :infinity).
@spec samecol_check(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
@samecol(expr): the current token's column must exactly match the enclosing reference column.
@spec skip_leading_trivia(tuple(), non_neg_integer(), atom() | nil) :: non_neg_integer()
Skips tokens named skip_name at pos (mirrors Grammar.VM's own
private skip_leading_trivia/3, for the same run_sequence case:
between two top-level matches there's no enclosing sequence for
Aether.Parser's skip-splicing to have spliced anything into, so a
leading @skip token here is genuinely unconsumed leftover from the
previous top-level match). skip_name is nil for a @noskip
grammar, in which case pos is returned unchanged.
@spec star(rule_fun(), tuple(), non_neg_integer(), [integer()], term(), captures()) :: rule_result()
Zero or more, greedy. Guards against a body that matches without
consuming anything (Grammar.VM.Compiler's own test_progress doc
explains why this can't be left to the static lint alone) -- a
zero-width success is still incorporated once, then the loop stops
rather than looping forever.
@spec token_at(tuple(), non_neg_integer()) :: Grammar.VM.Token.t() | nil
@spec try_alts([rule_fun()], tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
Ordered PEG choice: the first alternative that succeeds wins, tried in order.
@spec unexpected_token_error(tuple(), non_neg_integer(), String.t()) :: Ichor.Error.t()
A token stream matched only a prefix of the input -- the same error Grammar.VM's own match/2 reports.