final List< Token > tokens
The collected tokens.
static final byte INTERNAL_FIRST
The first id that can be used for internal state in a token marker.
This package contains the other part of the script editor.
void addToken(final int length, final byte id)
Adds a token to the token list.
void setToken(final byte token)
boolean isNextLineRequested()
Returns true if the next line should be repainted.
static final byte INTERNAL_LAST
The last id that can be used for internal state in a token marker.
void insertLines(final int index, final int lines)
Informs the token marker that lines have been inserted into the document.
Stores information about tokenized lines.
void ensureCapacity(final int index)
Ensures that the.
LineInfo[] lineInfo
An array for storing information about lines.
boolean nextLineRequested
True if the next line should be painted.
A token marker that splits lines of text into tokens.
abstract byte markTokensImpl(byte token, @NotNull Segment line)
An abstract method that splits a line up into tokens.
int length
The number of lines in the model being tokenized.
void deleteLines(final int index, final int lines)
Informs the token marker that line have been deleted from the document.
List< Token > markTokens(@NotNull final Segment line, final int lineIndex)
A wrapper for the lower-level.