Package com.sonar.sslr.api

Class Preprocessor

java.lang.Object
com.sonar.sslr.api.Preprocessor
@Deprecated public abstract class Preprocessor extends Object
Deprecated.
in 1.20, use your own preprocessor API instead.

A preprocessor is a component which can alter the stream of Token and Trivia.
The supported operations are injection and deletion. As Token are immutable, modification can be achieved by performing a deletion and an injection.

Without any preprocessor, the components of SSLR are chained as follows: 

 ---------------    ---------    ----------    -------
 | Source file | -> | Lexer | -> | Parser | -> | AST |
 ---------------    ---------    ----------    -------

A preprocessor sits between the Lexer and the Parser, thus components are chained as follows: 

 ---------------    ---------    ----------------    ----------    -------
 | Source file | -> | Lexer | -> | Preprocessor | -> | Parser | -> | AST |
 ---------------    ---------    ----------------    ----------    -------

Preprocessors can also be chained, in which case the stream of tokens seen by the second preprocessor is the stream of token which was generated by the first one: 

        ------------------    ------------------
 ... -> | Preprocessor 1 | -> | Preprocessor 2 | -> ...
        ------------------    ------------------

The first preprocessor sees the tokens as they were generated by the Lexer. The Parser sees the tokens as they were generated by the last preprocessor.

  • Constructor Details

    • Preprocessor

      public Preprocessor()
      Deprecated.

  • Method Details

    • init

      public void init()
      Deprecated.
      Method called before the lexing starts which can be overridden to initialize a state for instance.

    • process

      public abstract PreprocessorAction process(List<Token> tokens)
      Deprecated.

      Method called on each token seen by the current preprocessor.

      On each invocation of this method, all the remaining tokens are available. It is therefore possible to launch a Parser on the remaining input, and perform actions depending on the result of that parse. At the same time, a preprocessor can keep track of a state to keep track of the previous tokens which were seen, together with the init method.

      As an example, if a no-operation preprocessor (i.e. one which does not perform any injection nor deletion) is put after a Lexer which returned the tokens "a", "b", "c" and "EOF", this method will be called 4 times with different values of the tokens parameter:

      1. "a", "b", "c" and "EOF"
      2. "b", "c" and "EOF"
      3. "c" and "EOF"
      4. "EOF"

      All the operations to be performed by the preprocessor should be encapsulated in the PreprocessorAction return value object. The tokens list parameter is immutable.

      Parameters:
      tokens - An unmodifiable list of the remaining tokens.
      Returns:
      A preprocessor action, containing the Token/Trivia injections and deletions to perform. See PreprocessorAction for details.