Ichor.EBNF.ISO (Ichor v0.2.0)

Copy Markdown View Source

The ISO/IEC 14977 EBNF front-end: priv/grammar/ebnf-iso.aether parsed by Aether's own front-end, compiled by the native codegen backend (use Ichor), dispatching to Ichor.EBNF.ISO.Actions (Grammar.IR as its target category).

The .aether source lives under priv/grammar/, not lib/, for the same reason as Ichor.ABNF's own: use Ichor only ever reads it once, at compile time.

iex> {:ok, ruleset} = Ichor.EBNF.ISO.run("number = digit, { digit } ;\n")
iex> ruleset[:number]
%Grammar.IR.Seq{exprs: [%Grammar.IR.RuleRef{name: :digit}, %Grammar.IR.Star{expr: %Grammar.IR.RuleRef{name: :digit}}]}

run/1 (generated by use Ichor) returns {:ok, %{meta_identifier_atom => Grammar.IR.expr()}} for a whole syntax -- every rule the source defines. parse/1 and tokenize/1 (also generated) are available for a bare recognizer or raw token stream, with no Ichor.EBNF.ISO.Actions involved.

W3C-only syntax (?/*/+ suffixes, [...] as a character class rather than ISO's own "optional", ::=) fails informatively, not silently, when fed to this path: it's simply not valid ISO EBNF syntax, so Aether.Parser itself reports a normal, location-annotated Ichor.Error before Ichor.EBNF.ISO.Actions ever runs.

Summary

Functions

parse(input, context \\ nil)

Matches input against the grammar's root rule, requiring the entire (tokenized) input to be consumed. A bare recognizer -- no Ichor.Actions involved. context is read-only and only ever consulted by a Grammar.IR.Custom @native(...) node, if the grammar has one.

run(input, initial_context \\ nil)

Like parse/1, but runs the match through the compile-time-known Actions module, returning the grammar's actual evaluated result.

run_sequence(input, initial_context)

Like run/2, but evaluates input as a sequence of top-level matches against the root rule, one after another, threading context from each into the next (loading a standard-library file one top-level form at a time is the motivating case -- most grammars only ever need run/2).

tokenize(input, context \\ nil)

Tokenizes input completely via maximal munch, applies any @keywords/@refine rules, or reports the first position nothing matches. context is read-only and only ever consulted by a Grammar.IR.CustomLexeme @native(...) token, if the grammar has one.

Functions

parse(input, context \\ nil)

@spec parse(String.t(), term()) ::
  {:ok, non_neg_integer(), map()} | {:error, Ichor.Error.t()}

Matches input against the grammar's root rule, requiring the entire (tokenized) input to be consumed. A bare recognizer -- no Ichor.Actions involved. context is read-only and only ever consulted by a Grammar.IR.Custom @native(...) node, if the grammar has one.

run(input, initial_context \\ nil)

@spec run(String.t(), Ichor.Actions.context()) ::
  {:ok, term()} | {:error, Ichor.Error.t() | [Ichor.Error.t()]}

Like parse/1, but runs the match through the compile-time-known Actions module, returning the grammar's actual evaluated result.

run_sequence(input, initial_context)

@spec run_sequence(String.t(), Ichor.Actions.context()) ::
  {:ok, [term()], Ichor.Actions.context()}
  | {:error, Ichor.Error.t() | [Ichor.Error.t()]}

Like run/2, but evaluates input as a sequence of top-level matches against the root rule, one after another, threading context from each into the next (loading a standard-library file one top-level form at a time is the motivating case -- most grammars only ever need run/2).

tokenize(input, context \\ nil)

@spec tokenize(String.t(), term()) ::
  {:ok, [Grammar.VM.Token.t()]} | {:error, Ichor.Error.t()}

Tokenizes input completely via maximal munch, applies any @keywords/@refine rules, or reports the first position nothing matches. context is read-only and only ever consulted by a Grammar.IR.CustomLexeme @native(...) token, if the grammar has one.