Class Lexer<T extends Supplier<String>>

java.lang.Object

com.hypixel.hytale.server.npc.util.expression.compile.Lexer<T>

public class Lexer<T extends Supplier<String>>
extends Object

Break a character string into tokens.

The lexer uses fixed code to tokenize identifiers, numbers and strings. Operators and other symbols are parsed using a graph structure built from a table of tokens. Each node of this graph has a character to match, a token value if the path to the node represents a valid token and a list of possible follower nodes. The lexer for an expression is only built once and then considered read only. Tokenising a string requires a LexerContext to keep position and tokens. The lexer skips white space between tokens.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	INVALID_CHARACTER_IN_EXPRESSION
static final String	INVALID_NUMBER_FORMAT
static final String	UNTERMINATED_STRING

Constructor Summary

Constructors

Constructor	Description
Lexer(T tokenEnd, T tokenIdent, T tokenString, T tokenNumber, Stream<T> operators)	Create a lexer from table of operators.

Method Summary

Modifier and Type	Method	Description
T	nextToken(LexerContext<T> context)	Return next token in string.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

UNTERMINATED_STRING

public static final String UNTERMINATED_STRING

See Also:

• Constant Field Values

INVALID_NUMBER_FORMAT

public static final String INVALID_NUMBER_FORMAT

See Also:

• Constant Field Values

INVALID_CHARACTER_IN_EXPRESSION

public static final String INVALID_CHARACTER_IN_EXPRESSION

See Also:

• Constant Field Values

Constructor Details

Lexer

public Lexer(T tokenEnd,
 T tokenIdent,
 T tokenString,
 T tokenNumber,
 @Nonnull
 Stream<T> operators)

Create a lexer from table of operators.

Parameters:

tokenEnd - Value to return at string end

tokenIdent - Value to return for identifier

tokenString - Value to return for a string

tokenNumber - Value to return for a (floating point) number

operators - Stream of tokens to create lookup graph

Method Details

nextToken

public T nextToken(@Nonnull
 LexerContext<T> context)
            throws ParseException

Return next token in string.

Parameters:

context - Initialised LexerContext

Returns:

Token value (tokenEnd at end of string)

Throws:

ParseException - when no valid token was found