static final byte INTERNAL_FIRST
The first id that can be used for internal state in a token marker.
String toString()
Returns a string representation of this token.
static final byte LITERAL2
Literal 2 token id.
static final byte ID_COUNT
The total number of defined token ids.
static final byte INTERNAL_LAST
The last id that can be used for internal state in a token marker.
static final byte OPERATOR
Operator token id.
static final byte COMMENT2
Comment 2 token id.
void setId(final byte id)
Sets the id of this token.
int getLength()
Returns the length of this token.
static final byte INVALID
Invalid token id.
static final byte KEYWORD2
Keyword 2 token id.
static final byte KEYWORD1
Keyword 1 token id.
static final byte KEYWORD3
Keyword 3 token id.
byte getId()
Returns the id of this token.
static final byte LITERAL1
Literal 1 token id.
int length
The length of this token.
Token(final int length, final byte id)
Creates a new token.
static final byte COMMENT1
Comment 1 token id.
void setLength(final int length)
Sets the length of this token.