A linked list of tokens. More...

Collaboration diagram for net.sf.gridarta.textedit.textarea.Token:

Public Member Functions
byte	getId ()
	Returns the id of this token. More...

int	getLength ()
	Returns the length of this token. More...

void	setId (final byte id)
	Sets the id of this token. More...

void	setLength (final int length)
	Sets the length of this token. More...

	Token (final int length, final byte id)
	Creates a new token. More...

String	toString ()
	Returns a string representation of this token. More...

Static Public Attributes
static final byte	COMMENT1 = (byte) 1
	Comment 1 token id. More...

static final byte	COMMENT2 = (byte) 2
	Comment 2 token id. More...

static final byte	ID_COUNT = (byte) 11
	The total number of defined token ids. More...

static final byte	INTERNAL_FIRST = (byte) 100
	The first id that can be used for internal state in a token marker. More...

static final byte	INTERNAL_LAST = (byte) 126
	The last id that can be used for internal state in a token marker. More...

static final byte	INVALID = (byte) 10
	Invalid token id. More...

static final byte	KEYWORD1 = (byte) 6
	Keyword 1 token id. More...

static final byte	KEYWORD2 = (byte) 7
	Keyword 2 token id. More...

static final byte	KEYWORD3 = (byte) 8
	Keyword 3 token id. More...

static final byte	LABEL = (byte) 5
	Label token id. More...

static final byte	LITERAL1 = (byte) 3
	Literal 1 token id. More...

static final byte	LITERAL2 = (byte) 4
	Literal 2 token id. More...

static final byte	NULL = (byte) 0
	Normal text token id. More...

static final byte	OPERATOR = (byte) 9
	Operator token id. More...

Private Attributes
byte	id
	The id of this token. More...

int	length
	The length of this token. More...

Detailed Description

A linked list of tokens.

Each token has three fields - a token identifier, which is a byte value that can be looked up in the array returned by

SyntaxDocument.getColors()

to get a color value and a length value which is the length of the token in the text.

Author: Slava Pestov; Andreas Kirschbaum

Definition at line 21 of file Token.java.

Constructor & Destructor Documentation

◆ Token()

net.sf.gridarta.textedit.textarea.Token.Token	(	final int	length,
		final byte	id
	)

Creates a new token.

Parameters

length	the length of the token
id	the id of the token

Definition at line 116 of file Token.java.

References net.sf.gridarta.textedit.textarea.Token.id, and net.sf.gridarta.textedit.textarea.Token.length.

Member Function Documentation

◆ getId()

byte net.sf.gridarta.textedit.textarea.Token.getId ( )

Returns the id of this token.

Returns: the id of this token

Definition at line 133 of file Token.java.

References net.sf.gridarta.textedit.textarea.Token.id.

◆ getLength()

int net.sf.gridarta.textedit.textarea.Token.getLength ( )

Returns the length of this token.

Returns: the length of this token

Definition at line 149 of file Token.java.

References net.sf.gridarta.textedit.textarea.Token.length.

◆ setId()

void net.sf.gridarta.textedit.textarea.Token.setId ( final byte id )

Sets the id of this token.

Parameters

id	the new id of this token

Definition at line 141 of file Token.java.

References net.sf.gridarta.textedit.textarea.Token.id.

◆ setLength()

void net.sf.gridarta.textedit.textarea.Token.setLength ( final int length )

Sets the length of this token.

Parameters

length the new length of this token

Definition at line 157 of file Token.java.

References net.sf.gridarta.textedit.textarea.Token.length.

◆ toString()

String net.sf.gridarta.textedit.textarea.Token.toString ( )

Returns a string representation of this token.

Definition at line 125 of file Token.java.

Member Data Documentation

◆ COMMENT1

final byte net.sf.gridarta.textedit.textarea.Token.COMMENT1 = (byte) 1

static

Comment 1 token id.

This can be used to mark a comment.

Definition at line 31 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles(), net.sf.gridarta.textedit.textarea.tokenmarker.HTMLTokenMarker.markTokensImpl(), net.sf.gridarta.textedit.textarea.tokenmarker.PythonTokenMarker.markTokensImpl(), net.sf.gridarta.textedit.textarea.tokenmarker.LuaTokenMarker.markTokensImpl(), net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.markTokensImpl(), and net.sf.gridarta.textedit.textarea.tokenmarker.DaimoninAITokenMarker.markTokensImpl().

◆ COMMENT2

final byte net.sf.gridarta.textedit.textarea.Token.COMMENT2 = (byte) 2

static

Comment 2 token id.

This can be used to mark a comment.

Definition at line 36 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles(), net.sf.gridarta.textedit.textarea.tokenmarker.HTMLTokenMarker.markTokensImpl(), and net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.markTokensImpl().

◆ id

byte net.sf.gridarta.textedit.textarea.Token.id

private

The id of this token.

Definition at line 109 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.Token.getId(), net.sf.gridarta.textedit.textarea.Token.setId(), and net.sf.gridarta.textedit.textarea.Token.Token().

◆ ID_COUNT

final byte net.sf.gridarta.textedit.textarea.Token.ID_COUNT = (byte) 11

static

The total number of defined token ids.

Definition at line 89 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles().

◆ INTERNAL_FIRST

final byte net.sf.gridarta.textedit.textarea.Token.INTERNAL_FIRST = (byte) 100

static

The first id that can be used for internal state in a token marker.

Definition at line 94 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.tokenmarker.TokenMarker.addToken().

◆ INTERNAL_LAST

final byte net.sf.gridarta.textedit.textarea.Token.INTERNAL_LAST = (byte) 126

static

The last id that can be used for internal state in a token marker.

Definition at line 99 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.tokenmarker.TokenMarker.addToken().

◆ INVALID

final byte net.sf.gridarta.textedit.textarea.Token.INVALID = (byte) 10

static

Invalid token id.

This can be used to mark invalid or incomplete tokens, so the user can easily spot syntax errors.

Definition at line 84 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles(), net.sf.gridarta.textedit.textarea.tokenmarker.CrossfireDialogTokenMarker.markTokensImpl(), net.sf.gridarta.textedit.textarea.tokenmarker.HTMLTokenMarker.markTokensImpl(), net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.markTokensImpl(), net.sf.gridarta.textedit.textarea.tokenmarker.DaimoninAITokenMarker.markTokensImpl(), and net.sf.gridarta.textedit.textarea.TextAreaPainter.paintLine().

◆ KEYWORD1

final byte net.sf.gridarta.textedit.textarea.Token.KEYWORD1 = (byte) 6

static

Keyword 1 token id.

This can be used to mark a keyword. This should be used for general language constructs.

Definition at line 60 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.getCKeywords(), net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles(), net.sf.gridarta.textedit.textarea.tokenmarker.JavaScriptTokenMarker.getJavaScriptKeywords(), net.sf.gridarta.textedit.textarea.tokenmarker.LuaTokenMarker.getLuaKeywords(), net.sf.gridarta.textedit.textarea.tokenmarker.PythonTokenMarker.getPythonKeywords(), net.sf.gridarta.textedit.textarea.tokenmarker.CrossfireDialogTokenMarker.markTokensImpl(), and net.sf.gridarta.textedit.textarea.tokenmarker.HTMLTokenMarker.markTokensImpl().

◆ KEYWORD2

final byte net.sf.gridarta.textedit.textarea.Token.KEYWORD2 = (byte) 7

static

Keyword 2 token id.

This can be used to mark a keyword. This should be used for preprocessor commands, or variables.

Definition at line 66 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.getCKeywords(), net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles(), net.sf.gridarta.textedit.textarea.tokenmarker.PythonTokenMarker.getPythonKeywords(), net.sf.gridarta.textedit.textarea.tokenmarker.HTMLTokenMarker.markTokensImpl(), and net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.markTokensImpl().

◆ KEYWORD3

final byte net.sf.gridarta.textedit.textarea.Token.KEYWORD3 = (byte) 8

static

Keyword 3 token id.

This can be used to mark a keyword. This should be used for data types.

Definition at line 72 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.getCKeywords(), net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles(), net.sf.gridarta.textedit.textarea.tokenmarker.JavaScriptTokenMarker.getJavaScriptKeywords(), and net.sf.gridarta.textedit.textarea.tokenmarker.PythonTokenMarker.getPythonKeywords().

◆ LABEL

final byte net.sf.gridarta.textedit.textarea.Token.LABEL = (byte) 5

static

Label token id.

This can be used to mark labels (eg, C mode uses this to mark ...: sequences).

Definition at line 54 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles(), net.sf.gridarta.textedit.textarea.tokenmarker.JavaScriptTokenMarker.getJavaScriptKeywords(), and net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.markTokensImpl().

◆ length

int net.sf.gridarta.textedit.textarea.Token.length

private

The length of this token.

Definition at line 104 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.Token.getLength(), net.sf.gridarta.textedit.textarea.Token.setLength(), and net.sf.gridarta.textedit.textarea.Token.Token().

◆ LITERAL1

final byte net.sf.gridarta.textedit.textarea.Token.LITERAL1 = (byte) 3

static

Literal 1 token id.

This can be used to mark a string literal (eg, C mode uses this to mark "..." literals).

Definition at line 42 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles(), net.sf.gridarta.textedit.textarea.tokenmarker.CrossfireDialogTokenMarker.markTokensImpl(), net.sf.gridarta.textedit.textarea.tokenmarker.HTMLTokenMarker.markTokensImpl(), net.sf.gridarta.textedit.textarea.tokenmarker.PythonTokenMarker.markTokensImpl(), net.sf.gridarta.textedit.textarea.tokenmarker.LuaTokenMarker.markTokensImpl(), and net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.markTokensImpl().

◆ LITERAL2

final byte net.sf.gridarta.textedit.textarea.Token.LITERAL2 = (byte) 4

static

Literal 2 token id.

This can be used to mark an object literal (eg, Java mode uses this to mark true, false, etc).

Definition at line 48 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.getCKeywords(), net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles(), net.sf.gridarta.textedit.textarea.tokenmarker.LuaTokenMarker.getLuaKeywords(), net.sf.gridarta.textedit.textarea.tokenmarker.HTMLTokenMarker.markTokensImpl(), net.sf.gridarta.textedit.textarea.tokenmarker.PythonTokenMarker.markTokensImpl(), and net.sf.gridarta.textedit.textarea.tokenmarker.CTokenMarker.markTokensImpl().

◆ NULL

final byte net.sf.gridarta.textedit.textarea.Token.NULL = (byte) 0

static

Normal text token id.

This should be used to mark normal text.

Definition at line 26 of file Token.java.

◆ OPERATOR

final byte net.sf.gridarta.textedit.textarea.Token.OPERATOR = (byte) 9

static

Operator token id.

This can be used to mark an operator. (eg, SQL mode marks +, -, etc with this token type).

Definition at line 78 of file Token.java.

Referenced by net.sf.gridarta.textedit.textarea.SyntaxUtilities.getDefaultSyntaxStyles(), net.sf.gridarta.textedit.textarea.tokenmarker.LuaTokenMarker.getLuaKeywords(), net.sf.gridarta.textedit.textarea.tokenmarker.CrossfireDialogTokenMarker.markTokensImpl(), and net.sf.gridarta.textedit.textarea.tokenmarker.DaimoninAITokenMarker.markTokensImpl().

The documentation for this class was generated from the following file:

gridarta-gridarta/src/textedit/src/main/java/net/sf/gridarta/textedit/textarea/Token.java

Public Member Functions

Static Public Attributes

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ Token()

Member Function Documentation

◆ getId()

◆ getLength()

◆ setId()

◆ setLength()

◆ toString()

Member Data Documentation

◆ COMMENT1

◆ COMMENT2

◆ id

◆ ID_COUNT

◆ INTERNAL_FIRST

◆ INTERNAL_LAST

◆ INVALID

◆ KEYWORD1

◆ KEYWORD2

◆ KEYWORD3

◆ LABEL

◆ length

◆ LITERAL1

◆ LITERAL2

◆ NULL

◆ OPERATOR