View Source Renewex.Parser (renewex v0.12.0)

This module implements the core of the parser for reading Renew *.rnw files. The grammar to be used for parsing is defined by Renewex.Grammar.

The counter part for generating/serializing Renew *.rnw files is defined in Renewex.Serializer.

This Renewex.Parser manages the overall parser state and provides function to process the whole token stream coming from an *.rnw file.

Summary

Functions

The state of the parser consists of 4 fields

For a given list of tokens deduce that grammar version automatically and use it to parse the complete token stream.

Constructs a new parser state by taking a list of tokens. The grammar to be used for parsing Is extract from the version information encoded in the first token. If the first token is not an integer it is assumed that the oldest version of Renew had been used to create the file that this list of tokens are read from.

Expect the list of tokens to be empty and return the reversed list of parsed objects. If the list of tokens is not empty, return an error.

Get the version number of the parsers grammar.

Check if the list of tokens of the given parser state is empty.

Constructs a new parser state for a given list of tokens and a given grammar.

Parse the given list of tokens with the given grammar.

Given a parser state continue parsing by applying the given rule. The rule is the name of a java class defined in the parsers grammars hierarchy.

Parse a list, given the current state of a parser The next token must be an integer encoding the length of the list lst. The given function fun is a function that must parse single item of the list. It is applied lst times in a row.

Given a current parser state, read a single token of a given primitive type.

Given the current parser state, parse the object. The object is expected to be either NULL or a REF token or a serialized java class prefixed with its class name as next token. This prefix determines which grammar rule is used to continue parsing the list of tokens.

Given the current state of a parser skip the next token. If list of tokens is already empty return an error.

Given the current state of a parser expect the next tokens type to be one of the given types. Then skip it. If list of tokens already empty or the types do not match, return an error

Try to read multiple tokes of given types at once.

Skip a sequence of tokens of given skip_types.

Functions

Link to this function

%Renewex.Parser{}

View Source (struct)

The state of the parser consists of 4 fields:

  • grammar: The grammar definition to be used for serialization.
  • tokens: The list of yet to be processed tokens.
  • ref_stack: A stack (list) of already parsed Renewex.Storable.__struct__/0 structs. This is used to allow for Renewex.Storable.__struct__/0s to reference each other by index, as required by the Renew file format. Yet during parsing this stack is in reverse order (to allow fast pushes that the front of the list). It has be reversed for the referencing indices to be resolved correctly.
  • ref_count: The number of elements inside the ref_stack. This is used to to quick calculations without counting the elements in ref_stack.
Link to this function

detect_and_parse_document(tokens)

View Source

For a given list of tokens deduce that grammar version automatically and use it to parse the complete token stream.

Link to this function

detect_document_version(tokens)

View Source

Constructs a new parser state by taking a list of tokens. The grammar to be used for parsing Is extract from the version information encoded in the first token. If the first token is not an integer it is assumed that the oldest version of Renew had been used to create the file that this list of tokens are read from.

Expect the list of tokens to be empty and return the reversed list of parsed objects. If the list of tokens is not empty, return an error.

Get the version number of the parsers grammar.

Check if the list of tokens of the given parser state is empty.

Constructs a new parser state for a given list of tokens and a given grammar.

Link to this function

parse_document(tokens, version \\ Grammar.latest_version())

View Source

Parse the given list of tokens with the given grammar.

Link to this function

parse_grammar_rule(parser, rule)

View Source

Given a parser state continue parsing by applying the given rule. The rule is the name of a java class defined in the parsers grammars hierarchy.

In contrast to parse_storable no "NULL" or "REF" values are allowed and the parsed objects must not be prefixed with its class name. Instead the given rule explicitly determines which grammar rule to use.

Link to this function

parse_grammar_rule(parser, rule, storable)

View Source

Parse a list, given the current state of a parser The next token must be an integer encoding the length of the list lst. The given function fun is a function that must parse single item of the list. It is applied lst times in a row.

Link to this function

parse_primitive(parser, type)

View Source

Given a current parser state, read a single token of a given primitive type.

Link to this function

parse_storable(parser, expected_type \\ nil, return_ref \\ true)

View Source

Given the current parser state, parse the object. The object is expected to be either NULL or a REF token or a serialized java class prefixed with its class name as next token. This prefix determines which grammar rule is used to continue parsing the list of tokens.

If a serialized java object is read, it is prepedned to the ref_map of the parser. You can choose for this function to then eiter return a reference into this ref_map as integer (default) or to return the parsed object Storable struct itself (set return_ref to true).

When expected_type is given the parsed object is required to be a subtype of the expected_type accordings to the grammars class hierarhy.

Given the current state of a parser skip the next token. If list of tokens is already empty return an error.

Link to this function

skip_any(parser, of_types)

View Source

Given the current state of a parser expect the next tokens type to be one of the given types. Then skip it. If list of tokens already empty or the types do not match, return an error

Link to this function

try_parse(parser, types)

View Source

Try to read multiple tokes of given types at once.

Parameters

  • parser: The current parser state
  • types: A list of token types to read on sequence.

Returns

Either {tokens, new_parser} with tokens being a list of successfully read tokens. Or {:none, parser} if the expected sequency of tokens could not been read.

Link to this function

try_skip(err, skip_types)

View Source

Skip a sequence of tokens of given skip_types.