The interpreted runtime backend: compiles an Aether.Grammar to
bytecode and runs it against real input. No compile-time codegen step
-- give it a grammar (from Aether.Parser.parse/2 +
Grammar.Analysis.run/1) whenever you have one, including one your
own program only learns about at runtime (a user-supplied grammar, a
plugin, a REPL's :load). See the tutorial's "Loading a grammar at
runtime" section for a full worked example, and "Which path is right
for you?" for how this compares to use Ichor/Mix.Tasks.Ichor.Gen.
Two genuinely separate compiled programs are involved, matching
Aether's own "every grammar compiles to a Lexer feeding a Parser, never
a scannerless single pass" design: Grammar.VM.CharCompiler +
Grammar.VM.Tokenizer turn the input string into a token stream (maximal
munch over every declared token); Grammar.VM.RuleCompiler +
Grammar.VM.TokenInterpreter then run the rules over that stream,
never over raw characters -- building a raw capture tree as they go.
parse/3 is a bare recognizer (does grammar.root match input, full
stop) with no Ichor.Actions involved. run/4 is the actions-aware
entry point: it matches, then hands the raw capture tree to
Ichor.Actions.evaluate/5 for a given Actions module and initial
context, producing the grammar's actual result (a sandboxed program's
value, a config map, ...) instead of just a yes/no. run_sequence/4
is for a source that's a sequence of top-level matches against
grammar.root rather than one single match spanning the whole input
(loading a standard-library file one top-level form at a time, each one
threading context into the next, is the motivating case) -- most
grammars only ever need run/4.
context reaches the match phase too now (parse/3's third
argument, run/4's initial_context, run_sequence/4's
per-form-accumulated context for rule-level matching), read-only --
the one thing that ever consults it there is a Grammar.IR.Custom
@native(...) node, via Ichor.CustomRule.match/4, or a
Grammar.IR.CustomLexeme one via Ichor.CustomLexeme.scan/3. Nothing
else in Grammar.VM.TokenInterpreter/Grammar.VM.Tokenizer reads it; only
Ichor.Actions.evaluate/5 ever produces a new one.
One real gap for CustomLexeme specifically: run_sequence/4
tokenizes the entire input once, up front, with only
initial_context -- unlike rule-level matching (re-run per top-level
form, seeing that form's own accumulated context), a token that reads
context can't see anything a later form's own evaluation produced.
Fine for heredocs/string-interpolation (context there is either unused
or fixed for the whole file); a hazard only for something
\catcode-like that needs re-tokenization as context evolves
mid-sequence -- not something run_sequence/4 supports today.
Summary
Functions
Compiles grammar and matches it against input, requiring the root
rule to consume the entire (tokenized) input. Returns the number of
tokens consumed on success -- a bare recognizer result, no
Ichor.Actions involved.
Like parse/2, but runs the match through actions_module (a
Ichor.Actions implementation) starting from initial_context,
returning the grammar's actual evaluated result.
Evaluates input as a sequence of top-level matches against
grammar.root, one after another (skipping only the grammar's own
@skip trivia between them, the same as it would between any two
ordinary tokens), threading context from each into the next. Returns
every top-level value, in order, plus the final context.
Functions
@spec parse(Aether.Grammar.t(), String.t(), term()) :: {:ok, non_neg_integer()} | {:error, Ichor.Error.t()}
Compiles grammar and matches it against input, requiring the root
rule to consume the entire (tokenized) input. Returns the number of
tokens consumed on success -- a bare recognizer result, no
Ichor.Actions involved.
@spec run(Aether.Grammar.t(), String.t(), module(), Ichor.Actions.context()) :: {:ok, term()} | {:error, Ichor.Error.t() | [Ichor.Error.t()]}
Like parse/2, but runs the match through actions_module (a
Ichor.Actions implementation) starting from initial_context,
returning the grammar's actual evaluated result.
@spec run_sequence(Aether.Grammar.t(), String.t(), module(), Ichor.Actions.context()) :: {:ok, [term()], Ichor.Actions.context()} | {:error, Ichor.Error.t() | [Ichor.Error.t()]}
Evaluates input as a sequence of top-level matches against
grammar.root, one after another (skipping only the grammar's own
@skip trivia between them, the same as it would between any two
ordinary tokens), threading context from each into the next. Returns
every top-level value, in order, plus the final context.