public static final class TokenChunker.Builder extends Object
token("A").ignore("A")
for the input string "A" will produce token, whereas
ignore("A").token("A")
will not.Modifier and Type | Method and Description |
---|---|
TokenChunker |
build() |
TokenChunker.Builder |
ignore(String regularExpression)
Defines that sequence of characters must be ignored, if it matches specified regular expression.
|
TokenChunker.Builder |
token(String regularExpression)
Defines that sequence of characters, which is matched specified regular expression, is a token.
|
TokenChunker.Builder |
token(String regularExpression,
String normalizationValue)
Defines that sequence of characters, which is matched specified regular expression, is a token with specified value.
|
public TokenChunker build()
public TokenChunker.Builder ignore(String regularExpression)
public TokenChunker.Builder token(String regularExpression)
public TokenChunker.Builder token(String regularExpression, String normalizationValue)
Copyright © 2009-2012 SonarSource. All Rights Reserved.