Ichor.BNF (Ichor v0.2.0)

Copy Markdown View Source

The classical BNF front-end -- no single citable standard exists; this follows the ALGOL 60 Report's own convention with quoted terminals, matching how BNF is actually written today (see Ichor.BNF.Actions's own moduledoc for the deliberate deviation that is): priv/grammar/bnf.aether parsed by Aether's own front-end, compiled by the native codegen backend (use Ichor), dispatching to Ichor.BNF.Actions (Grammar.IR as its target category).

The .aether source lives under priv/grammar/, not lib/, for the same reason as Ichor.ABNF's own: use Ichor only ever reads it once, at compile time.

iex> {:ok, ruleset} = Ichor.BNF.run("<digit> ::= '0' | '1'\n")
iex> ruleset[:digit]
%Grammar.IR.Choice{exprs: [%Grammar.IR.Literal{value: "0"}, %Grammar.IR.Literal{value: "1"}]}

run/1 (generated by use Ichor) returns {:ok, %{nonterminal_name_atom => Grammar.IR.expr()}} for a whole grammar_file -- every rule the source defines, keyed by its nonterminal name exactly as written (classical BNF has no case-insensitivity rule the way ABNF does). parse/1 and tokenize/1 (also generated) are available for a bare recognizer or raw token stream, with no Ichor.BNF.Actions involved.

Summary

Functions

parse(input, context \\ nil)

Matches input against the grammar's root rule, requiring the entire (tokenized) input to be consumed. A bare recognizer -- no Ichor.Actions involved. context is read-only and only ever consulted by a Grammar.IR.Custom @native(...) node, if the grammar has one.

run(input, initial_context \\ nil)

Like parse/1, but runs the match through the compile-time-known Actions module, returning the grammar's actual evaluated result.

run_sequence(input, initial_context)

Like run/2, but evaluates input as a sequence of top-level matches against the root rule, one after another, threading context from each into the next (loading a standard-library file one top-level form at a time is the motivating case -- most grammars only ever need run/2).

tokenize(input, context \\ nil)

Tokenizes input completely via maximal munch, applies any @keywords/@refine rules, or reports the first position nothing matches. context is read-only and only ever consulted by a Grammar.IR.CustomLexeme @native(...) token, if the grammar has one.

Functions

parse(input, context \\ nil)

@spec parse(String.t(), term()) ::
  {:ok, non_neg_integer(), map()} | {:error, Ichor.Error.t()}

Matches input against the grammar's root rule, requiring the entire (tokenized) input to be consumed. A bare recognizer -- no Ichor.Actions involved. context is read-only and only ever consulted by a Grammar.IR.Custom @native(...) node, if the grammar has one.

run(input, initial_context \\ nil)

@spec run(String.t(), Ichor.Actions.context()) ::
  {:ok, term()} | {:error, Ichor.Error.t() | [Ichor.Error.t()]}

Like parse/1, but runs the match through the compile-time-known Actions module, returning the grammar's actual evaluated result.

run_sequence(input, initial_context)

@spec run_sequence(String.t(), Ichor.Actions.context()) ::
  {:ok, [term()], Ichor.Actions.context()}
  | {:error, Ichor.Error.t() | [Ichor.Error.t()]}

Like run/2, but evaluates input as a sequence of top-level matches against the root rule, one after another, threading context from each into the next (loading a standard-library file one top-level form at a time is the motivating case -- most grammars only ever need run/2).

tokenize(input, context \\ nil)

@spec tokenize(String.t(), term()) ::
  {:ok, [Grammar.VM.Token.t()]} | {:error, Ichor.Error.t()}

Tokenizes input completely via maximal munch, applies any @keywords/@refine rules, or reports the first position nothing matches. context is read-only and only ever consulted by a Grammar.IR.CustomLexeme @native(...) token, if the grammar has one.