Rule-level helpers shared by every Grammar.Native.RuleCompiler-generated
function -- the Parser stage of Aether's Reader/Tokenizer/Lexer/Parser
split. Mirrors Grammar.VM.TokenInterpreter's own semantics exactly
(same backtracking discipline, same no-progress guard on Star), just
expressed as plain function composition instead of a bytecode
interpreter loop -- Grammar.VM.Compiler's moduledoc describes the
same combinators this module implements.
Every combinator here operates on a {:ok, pos, ref_stack, captures} | :fail shape, where captures is a raw-captures list in exactly the
shape Ichor.Actions already expects (Ichor.Capture.raw_captures/0:
{:token, name, text} / {:rule, name, sub_captures} / {:text, text} values, keyed by name, in first-occurrence RHS order) --
generated code builds this list compositionally as it goes, rather
than through a single mutable per-rule frame the way
Grammar.VM.TokenInterpreter does, since plain Elixir recursion
already gives every failed alternative's partial captures nowhere to
leak to (they're just a discarded return value, never mutated shared
state). A list, not a map: list order is the one thing Elixir
actually guarantees, which is exactly what lets Ichor.Actions.eval_all/2
evaluate sibling captures in true source order instead of trusting a
plain map's own (cross-OTP-version-unstable) iteration order.
Summary
Functions
Positive lookahead: consumes nothing either way.
Calls fun exactly n times in sequence, merging captures, failing outward if any call fails.
Concatenates the raw text of every token in [start_pos, end_pos) -- the :text capture kind's raw value.
@indent(expr): the current token's column must exceed the enclosing reference column; pushes its own column as the new reference for expr, pops it back off afterward.
Matches one stream token named name at pos, advancing by one -- the
raw building block every bare token reference compiles to. capture
is the token's own override (see Grammar.VM.Token) -- nil unless a
Grammar.IR.CustomLexeme matched it with one attached.
Folds every entry of new into acc, list-appending (or
list-wrapping) on repeat capture names -- the same rule a rule's own
capture frame follows in the VM (Grammar.VM.TokenInterpreter's own
private merge_capture helper). A new name is appended at the end of
acc (landing at its first-occurrence position); a repeat name
updates its existing entry in place, not moving it.
Negative lookahead: consumes nothing either way.
Zero or one: on failure, a zero-width success with no captures.
One or more, greedy: one mandatory match, then star/6 for the rest.
Bounded repetition {n,m}: min mandatory calls, then up to max - min more (or unbounded, when max == :infinity).
@samecol(expr): the current token's column must exactly match the enclosing reference column.
Skips tokens named skip_name at pos (mirrors Grammar.VM's own
private skip_leading_trivia/3, for the same run_sequence case:
between two top-level matches there's no enclosing sequence for
Aether.Parser's skip-splicing to have spliced anything into, so a
leading @skip token here is genuinely unconsumed leftover from the
previous top-level match). skip_name is nil for a @noskip
grammar, in which case pos is returned unchanged.
Zero or more, greedy. Guards against a body that matches without
consuming anything (Grammar.VM.Compiler's own test_progress doc
explains why this can't be left to the static lint alone) -- a
zero-width success is still incorporated once, then the loop stops
rather than looping forever.
Ordered PEG choice: the first alternative that succeeds wins, tried in order.
A token stream matched only a prefix of the input -- the same error Grammar.VM's own match/2 reports.
Types
@type captures() :: Ichor.Capture.raw_captures()
@type rule_fun() :: (tuple(), non_neg_integer(), [integer()], term() -> rule_result())
@type rule_result() :: {:ok, non_neg_integer(), [integer()], captures()} | :fail
Functions
@spec and_pred(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
Positive lookahead: consumes nothing either way.
@spec call_n_times( rule_fun(), non_neg_integer(), tuple(), non_neg_integer(), [integer()], term() ) :: rule_result()
Calls fun exactly n times in sequence, merging captures, failing outward if any call fails.
@spec concat_text(tuple(), non_neg_integer(), non_neg_integer()) :: String.t()
Concatenates the raw text of every token in [start_pos, end_pos) -- the :text capture kind's raw value.
@spec indent_enter(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
@indent(expr): the current token's column must exceed the enclosing reference column; pushes its own column as the new reference for expr, pops it back off afterward.
@spec match_token(tuple(), non_neg_integer(), atom()) :: {:ok, non_neg_integer(), String.t(), term()} | :fail
Matches one stream token named name at pos, advancing by one -- the
raw building block every bare token reference compiles to. capture
is the token's own override (see Grammar.VM.Token) -- nil unless a
Grammar.IR.CustomLexeme matched it with one attached.
Folds every entry of new into acc, list-appending (or
list-wrapping) on repeat capture names -- the same rule a rule's own
capture frame follows in the VM (Grammar.VM.TokenInterpreter's own
private merge_capture helper). A new name is appended at the end of
acc (landing at its first-occurrence position); a repeat name
updates its existing entry in place, not moving it.
Unlike that VM function, this one has to merge whole subtree
capture lists at once (a Seq/Star node's already-accumulated
captures), not a single new occurrence at a time -- so a value that's
already a list (built up by a nested Star/Plus/Rep sharing this
same capture name) has to be flattened into the result rather than
wrapped as one more element, or [a, b, c] merged against a sibling
d would wrongly become [d, [a, b, c]] instead of [d, a, b, c].
A raw capture value is always a {:token, ...} / {:rule, ...} /
{:text, ...} tuple, never a bare list on its own, so "the value is
a list" unambiguously means "already an accumulation," never "one
occurrence that happens to look like a list."
@spec not_pred(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
Negative lookahead: consumes nothing either way.
@spec opt(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
Zero or one: on failure, a zero-width success with no captures.
@spec plus(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
One or more, greedy: one mandatory match, then star/6 for the rest.
@spec rep( rule_fun(), non_neg_integer(), non_neg_integer() | :infinity, tuple(), non_neg_integer(), [integer()], term() ) :: rule_result()
Bounded repetition {n,m}: min mandatory calls, then up to max - min more (or unbounded, when max == :infinity).
@spec samecol_check(rule_fun(), tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
@samecol(expr): the current token's column must exactly match the enclosing reference column.
@spec skip_leading_trivia(tuple(), non_neg_integer(), atom() | nil) :: non_neg_integer()
Skips tokens named skip_name at pos (mirrors Grammar.VM's own
private skip_leading_trivia/3, for the same run_sequence case:
between two top-level matches there's no enclosing sequence for
Aether.Parser's skip-splicing to have spliced anything into, so a
leading @skip token here is genuinely unconsumed leftover from the
previous top-level match). skip_name is nil for a @noskip
grammar, in which case pos is returned unchanged.
@spec star(rule_fun(), tuple(), non_neg_integer(), [integer()], term(), captures()) :: rule_result()
Zero or more, greedy. Guards against a body that matches without
consuming anything (Grammar.VM.Compiler's own test_progress doc
explains why this can't be left to the static lint alone) -- a
zero-width success is still incorporated once, then the loop stops
rather than looping forever.
@spec token_at(tuple(), non_neg_integer()) :: Grammar.VM.Token.t() | nil
@spec try_alts([rule_fun()], tuple(), non_neg_integer(), [integer()], term()) :: rule_result()
Ordered PEG choice: the first alternative that succeeds wins, tried in order.
@spec unexpected_token_error(tuple(), non_neg_integer(), String.t()) :: Ichor.Error.t()
A token stream matched only a prefix of the input -- the same error Grammar.VM's own match/2 reports.