int length
The length of this token.
static final byte COMMENT2
Comment 2 token id.
static final byte INTERNAL_FIRST
The first id that can be used for internal state in a token marker.
Token(final int length, final byte id)
Creates a new token.
void setId(final byte id)
Sets the id of this token.
byte getId()
Returns the id of this token.
static final byte COMMENT1
Comment 1 token id.
static final byte LITERAL1
Literal 1 token id.
String toString()
Returns a string representation of this token.
static final byte KEYWORD3
Keyword 3 token id.
void setLength(final int length)
Sets the length of this token.
static final byte OPERATOR
Operator token id.
static final byte LITERAL2
Literal 2 token id.
int getLength()
Returns the length of this token.
static final byte KEYWORD2
Keyword 2 token id.
static final byte INTERNAL_LAST
The last id that can be used for internal state in a token marker.
static final byte KEYWORD1
Keyword 1 token id.
static final byte ID_COUNT
The total number of defined token ids.
static final byte INVALID
Invalid token id.