View Source Renewex.Parser (renewex v0.6.0)

This module implements the core of the parser for reading Renew *.rnw files. The grammar to be used for parsing is defined by Renewex.Grammar.

This Renewex.Parser manages the overall parser state and provides function to process the whole token stream coming from an *.rnw file.

Summary

Functions

For a given list of tokens deduce that grammar version automatically and use it to parse the complete token stream.

Constructs a new parser state by taking a list of tokens. The grammar to be used for parsing Is extract from the version information encoded in the first token. If the first token is not an integer it is assumed that the oldest version of Renew had been used to create the file that this list of tokens are read from.

Expect the list of tokens to be empty and return the reversed list of parsed objects. If the list of tokens is not empty, return an error.

Check if the list of tokens of the given parser state is empty.

Constructs a new parser state for a given list of tokens and a given grammar.

Parse the given list of tokens with the given grammar.

Given a parser state continue parsing by applying the given rule. The rule is the name of a java class defined in the parsers grammars hierarchy.

Given the current state of a parser parse a list. The next token must be an integer encoding the length of the list (l). The given function fun is a function that must parse single item of the list. It is applied l times in a row.

Given a current parser state, read a single token of a given primitive type.

Given the current parser state, parse the object. The object is expected to be either NULL or a REF token or a serialized java class prefixed with its class name as next token. This prefix determines which grammar rule is used to continue parsing the list of tokens.

Given the current state of a parser skip the next token. If list of tokens is already empty return an error.

Given the current state of a parser expect the next tokens type to be one of the given types. Then skip it. If list of tokens already empty or the types do not match, return an error

Given the current state of a parser try to skip multiple tokens of the expected types. If n types are given and the next n tokens match these types, the tokens are skipped. If anything does not match the original parser state is returned.

Functions

Link to this function

detect_and_parse_document(tokens)

View Source

For a given list of tokens deduce that grammar version automatically and use it to parse the complete token stream.

Link to this function

detect_document_version(tokens)

View Source

Constructs a new parser state by taking a list of tokens. The grammar to be used for parsing Is extract from the version information encoded in the first token. If the first token is not an integer it is assumed that the oldest version of Renew had been used to create the file that this list of tokens are read from.

Expect the list of tokens to be empty and return the reversed list of parsed objects. If the list of tokens is not empty, return an error.

Check if the list of tokens of the given parser state is empty.

Constructs a new parser state for a given list of tokens and a given grammar.

Link to this function

parse_document(tokens, version \\ Grammar.latest_version())

View Source

Parse the given list of tokens with the given grammar.

Link to this function

parse_grammar_rule(parser, rule)

View Source

Given a parser state continue parsing by applying the given rule. The rule is the name of a java class defined in the parsers grammars hierarchy.

In contrast to parse_storable no NULL or REF values are allowed and the parsed objects must not be prefixed with its class name. Instead the given rule explicitly determines which grammar rule to use.

Link to this function

parse_grammar_rule(parser, rule, storable)

View Source

Given the current state of a parser parse a list. The next token must be an integer encoding the length of the list (l). The given function fun is a function that must parse single item of the list. It is applied l times in a row.

Link to this function

parse_primitive(parser, type)

View Source

Given a current parser state, read a single token of a given primitive type.

Link to this function

parse_storable(parser, expected_type \\ nil, return_ref \\ true)

View Source

Given the current parser state, parse the object. The object is expected to be either NULL or a REF token or a serialized java class prefixed with its class name as next token. This prefix determines which grammar rule is used to continue parsing the list of tokens.

If a serialized java object is read, it is prepedned to the ref_map of the parser. You can choose for this function to then eiter return a reference into this ref_map as integer (default) or to return the parsed object Storable struct itself (set return_ref to true).

When expected_type is given the parsed object is required to be a subtype of the expected_type accordings to the grammars class hierarhy.

Given the current state of a parser skip the next token. If list of tokens is already empty return an error.

Link to this function

skip_any(parser, of_types)

View Source

Given the current state of a parser expect the next tokens type to be one of the given types. Then skip it. If list of tokens already empty or the types do not match, return an error

Given the current state of a parser try to skip multiple tokens of the expected types. If n types are given and the next n tokens match these types, the tokens are skipped. If anything does not match the original parser state is returned.

Another of try_skip that takes not just a parser but a tupkle of {:ok, result, parser} or {:error, result, parser}. If {:error, ...} simply return the input. If {:ok, result, parser} apply try_skip to the parser.