Grammar.VM.RuleCompiler (Ichor v0.2.1)

Copy Markdown View Source

Compiles every rule in a grammar into one linked Grammar.VM.Program, run by Grammar.VM.TokenInterpreter against the lexer's token stream (never against raw characters -- Aether's two-stage Lexer -> Parser split). A RuleRef compiles to {:token, name} (consume one stream token of that type) when it names a token, or {:call, name} (jump into that rule's own compiled code) when it names another rule -- the two are otherwise indistinguishable in the IR, so the grammar's own token/rule namespaces are what disambiguate.

Indent/@samecol read a token's column straight off the stream (the lexer already recorded it), rather than tracking columns incrementally during parsing.

Every bare RuleRef -- and every Capture -- is additionally bracketed with {:cap_start, ...}/{:cap_end, name}, so Grammar.VM.TokenInterpreter can build the raw capture tree Ichor.Actions needs: a bare reference is implicitly captured under its own name (this is what lets a calculator's own handle_rule(:expr, %{op: ops, term: terms}, ctx) see a :term key even though expr's grammar never writes term:term); an explicit name:expr additionally captures under the given name. A bare reference to an anonymous auto-promoted token (an inline literal like "(", never a name the grammar author chose) gets no implicit capture at all: writing a bare literal is precisely how a grammar author says "I don't care about this," and implicitly capturing it anyway would leak a compiler-generated name into every rule that uses one, breaking the "exactly one capture passes straight through" default fallback for something as simple as factor := NUMBER | "(" expr ")". Explicitly naming one (paren:"(") still works -- that's a deliberate capture, not an implicit one.

Only a capture whose expression is directly a bare RuleRef (no wrapping Choice/Star/... in between) dispatches to that rule/token's own action on eval -- anything else captures the raw matched text span instead. That's a deliberate, narrower rule than "figure out the one branch that actually matched at runtime": it keeps capture semantics decidable from the grammar's static shape, at the cost of not dispatching through a capture like x:(rule_a | rule_b) -- not needed by any of Ichor's own worked-example grammars, and worth revisiting only if a real grammar actually needs it.

capture_shapes/1 is a separate static pass answering "which capture names, for each rule, sit under a Star/Plus/Rep anywhere in that rule's body" -- a name captured that way needs a list value even when it happened to match once, or zero times (an absent key, vs. an empty list, isn't something a Calculator.Actions-style %{op: ops, factor: factors} pattern match can tell apart from "this rule shape doesn't have that capture at all"). Ichor.Actions uses this table to normalize the raw capture tree before dispatch.

Summary

Functions

For every rule, the set of capture names that sit under a Star/Plus/Rep somewhere in that rule's own body (not descending into other rules/tokens it calls).

Functions

capture_shapes(grammar)

@spec capture_shapes(Aether.Grammar.t()) :: %{required(atom()) => MapSet.t(atom())}

For every rule, the set of capture names that sit under a Star/Plus/Rep somewhere in that rule's own body (not descending into other rules/tokens it calls).

compile(grammar)

@spec compile(Aether.Grammar.t()) :: Grammar.VM.Program.t()