com.sonar.sslr.api

Class Preprocessor

java.lang.Object

com.sonar.sslr.api.Preprocessor

public abstract class Preprocessor
extends java.lang.Object

A preprocessor is a component which can alter the stream of Token and Trivia.
The supported operations are injection and deletion. As Token are immutable, modification can be achieved by performing a deletion and an injection.

Without any preprocessor, the components of SSLR are chained as follows:

 ---------------    ---------    ----------    -------
 | Source file | -> | Lexer | -> | Parser | -> | AST |
 ---------------    ---------    ----------    -------
 

A preprocessor sits between the Lexer and the Parser, thus components are chained as follows:

 ---------------    ---------    ----------------    ----------    -------
 | Source file | -> | Lexer | -> | Preprocessor | -> | Parser | -> | AST |
 ---------------    ---------    ----------------    ----------    -------
 

Preprocessors can also be chained, in which case the stream of tokens seen by the second preprocessor is the stream of token which was generated by the first one:

        ------------------    ------------------
 ... -> | Preprocessor 1 | -> | Preprocessor 2 | -> ...
        ------------------    ------------------
 

The first preprocessor sees the tokens as they were generated by the Lexer. The Parser sees the tokens as they were generated by the last preprocessor.

Constructor Summary

Constructors

Constructor and Description
Preprocessor()

Method Summary

Methods

Modifier and Type	Method and Description
void	init() Method called before the lexing starts which can be overridden to initialize a state for instance.
abstract PreprocessorAction	process(java.util.List<Token> tokens) Method called on each token seen by the current preprocessor.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Preprocessor

public Preprocessor()

Method Detail

init

public void init()

Method called before the lexing starts which can be overridden to initialize a state for instance.

process

public abstract PreprocessorAction process(java.util.List<Token> tokens)

Method called on each token seen by the current preprocessor.

On each invocation of this method, all the remaining tokens are available. It is therefore possible to launch a Parser on the remaining input, and perform actions depending on the result of that parse. At the same time, a preprocessor can keep track of a state to keep track of the previous tokens which were seen, together with the init method.

As an example, if a no-operation preprocessor (i.e. one which does not perform any injection nor deletion) is put after a Lexer which returned the tokens "a", "b", "c" and "EOF", this method will be called 4 times with different values of the tokens parameter:

1. "a", "b", "c" and "EOF"
2. "b", "c" and "EOF"
3. "c" and "EOF"
4. "EOF"

All the operations to be performed by the preprocessor should be encapsulated in the PreprocessorAction return value object. The tokens list parameter is immutable.

Parameters:

tokens - An unmodifiable list of the remaining tokens.

Returns:

A preprocessor action, containing the Token/Trivia injections and deletions to perform. See PreprocessorAction for details.