View Source Renewex.Parser (renewex v0.9.0)
This module implements the core of the parser for reading Renew *.rnw
files.
The grammar to be used for parsing is defined by Renewex.Grammar
.
The counter part for generating/serializing Renew *.rnw
files is defined in Renewex.Serializer
.
This Renewex.Parser
manages the overall parser state and provides function to process the whole
token stream coming from an *.rnw
file.
Summary
Functions
The state of the parser consists of 4 fields
For a given list of tokens deduce that grammar version automatically and use it to parse the complete token stream.
Constructs a new parser state by taking a list of tokens. The grammar to be used for parsing Is extract from the version information encoded in the first token. If the first token is not an integer it is assumed that the oldest version of Renew had been used to create the file that this list of tokens are read from.
Expect the list of tokens to be empty and return the reversed list of parsed objects. If the list of tokens is not empty, return an error.
Get the version number of the parsers grammar.
Check if the list of tokens of the given parser state is empty.
Constructs a new parser state for a given list of tokens and a given grammar.
Parse the given list of tokens with the given grammar.
Given a parser state continue parsing by applying the given rule. The rule is the name of a java class defined in the parsers grammars hierarchy.
Parse a list, given the current state of a parser
The next token must be an integer encoding the length of the list lst
.
The given function fun
is a function that must parse single item of the list.
It is applied lst
times in a row.
Given a current parser state, read a single token of a given primitive type.
Given the current parser state, parse the object. The object is expected to be either NULL or a REF token or a serialized java class prefixed with its class name as next token. This prefix determines which grammar rule is used to continue parsing the list of tokens.
Given the current state of a parser skip the next token. If list of tokens is already empty return an error.
Given the current state of a parser expect the next tokens type to be one of the given types. Then skip it. If list of tokens already empty or the types do not match, return an error
Try to read multiple tokes of given types
at once.
Skip a sequence of tokens of given skip_types
.
Functions
The state of the parser consists of 4 fields:
grammar
: The grammar definition to be used for serialization.tokens
: The list of yet to be processed tokens.ref_stack
: A stack (list) of already parsedRenewex.Storable.__struct__/0
structs. This is used to allow forRenewex.Storable.__struct__/0
s to reference each other by index, as required by the Renew file format. Yet during parsing this stack is in reverse order (to allow fast pushes that the front of the list). It has be reversed for the referencing indices to be resolved correctly.ref_count
: The number of elements inside theref_stack
. This is used to to quick calculations without counting the elements inref_stack
.
For a given list of tokens deduce that grammar version automatically and use it to parse the complete token stream.
Constructs a new parser state by taking a list of tokens. The grammar to be used for parsing Is extract from the version information encoded in the first token. If the first token is not an integer it is assumed that the oldest version of Renew had been used to create the file that this list of tokens are read from.
Expect the list of tokens to be empty and return the reversed list of parsed objects. If the list of tokens is not empty, return an error.
Get the version number of the parsers grammar.
Check if the list of tokens of the given parser state is empty.
Constructs a new parser state for a given list of tokens and a given grammar.
Parse the given list of tokens with the given grammar.
Given a parser state continue parsing by applying the given rule. The rule is the name of a java class defined in the parsers grammars hierarchy.
In contrast to parse_storable
no "NULL"
or "REF"
values are allowed and the parsed objects must
not be prefixed with its class name. Instead the given rule explicitly determines which grammar rule to use.
Parse a list, given the current state of a parser
The next token must be an integer encoding the length of the list lst
.
The given function fun
is a function that must parse single item of the list.
It is applied lst
times in a row.
Given a current parser state, read a single token of a given primitive type.
Given the current parser state, parse the object. The object is expected to be either NULL or a REF token or a serialized java class prefixed with its class name as next token. This prefix determines which grammar rule is used to continue parsing the list of tokens.
If a serialized java object is read, it is prepedned to the ref_map of the parser.
You can choose for this function to then eiter return a reference into this ref_map as integer (default)
or to return the parsed object Storable struct itself (set return_ref
to true).
When expected_type
is given the parsed object is required to be a subtype of the expected_type
accordings to the grammars
class hierarhy.
Given the current state of a parser skip the next token. If list of tokens is already empty return an error.
Given the current state of a parser expect the next tokens type to be one of the given types. Then skip it. If list of tokens already empty or the types do not match, return an error
Try to read multiple tokes of given types
at once.
Parameters
parser
: The current parser statetypes
: A list of token types to read on sequence.
Returns
Either {tokens, new_parser}
with tokens
being a list of successfully read tokens.
Or {:none, parser}
if the expected sequency of tokens could not been read.
Skip a sequence of tokens of given skip_types
.