org.apache.flex.compiler.internal.parsing

Class TokenBase

java.lang.Object

antlr.Token

org.apache.flex.compiler.internal.parsing.TokenBase

All Implemented Interfaces:

Cloneable, ISourceLocation, ICMToken

Direct Known Subclasses:

ASDocToken, ASToken, MetadataToken

public abstract class TokenBase
extends antlr.Token
implements ICMToken, ISourceLocation

Base class of ASToken, MXMLToken, CSSToken

Field Summary

Fields inherited from class antlr.Token

badToken, EOF_TYPE, INVALID_TYPE, MIN_USER_TYPE, NULL_TREE_LOOKAHEAD, SKIP, type

Fields inherited from interface org.apache.flex.compiler.common.ISourceLocation

UNKNOWN

Constructor Summary

Constructors

Constructor and Description
TokenBase(int tokenType, int start, int end, int line, int column, CharSequence text) Constructor
TokenBase(TokenBase o) Copy constructor

Method Summary

Methods

Modifier and Type	Method and Description
void	adjustLocation(int offsetAdjustment, int lineAdjustment, int columnAdjustment) Adjust all associated offsets by the adjustment amount
void	adjustOffsets(int offsetAdjustment) Adjust all associated offsets by the adjustment amount
int	getAbsoluteEnd() Gets the absolute starting offset.
int	getAbsoluteStart() Gets the absolute starting offset.
CharSequence	getCharSequence() Returns the underlying CharSequence that represents the contents of this token
int	getColumn() Returns the column of this token.
int	getEnd() Returns the end offset of this token
int	getEndColumn() Gets the local column number at the end.
int	getEndLine() Gets the local line number at the end.
int	getLine() Returns the line of this token..
int	getLocalEnd()
int	getLocalStart()
String	getSourcePath() Get the source file path of the node.
int	getStart() Returns the start offset of this token
String	getText() Get the text represented by this token
protected abstract String	getTypeString()
boolean	isImplicit() Determine whether or not this token is bogus (i.e. the start and end offsets are the same, which implies that it was inserted from an included file or during token fixup)
boolean	isLocked() Returns whether this token can be overwritten when it is a member of a token pool
void	lock() Locks this token.
boolean	matchesLine(TokenBase other)
void	reuse(int tokenType, int start, int end, int line, int column, CharSequence text)
void	setColumn(int column)
void	setEnd(int end)
void	setEndColumn(int column)
void	setEndLine(int line)
void	setLine(int line)
void	setLocation(int start, int end, int line, int column)
void	setSourcePath(String path)
void	setStart(int start)
void	setText(CharSequence text) Set the CharSequence represented by this token
void	setText(String text) Set the text represented by this token
void	storeLocalOffset() Capture the current start/end offsets as this token's local offsets.
String	toDumpString() For debugging only.
String	toString() For debugging only.
void	truncate(int trimLeft, int trimRight) Reduce the span of the token by removing characters from the beginning and end.
boolean	verify() Verifies that this token has its type and location information set.

Methods inherited from class antlr.Token

getFilename, getType, setFilename, setType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.flex.compiler.parsing.ICMToken

changeType, getType

Constructor Detail

TokenBase

public TokenBase(int tokenType,
         int start,
         int end,
         int line,
         int column,
         CharSequence text)

Constructor

Parameters:

tokenType - type of token

start - location location information

end - location location information

line - location location information

column - location location information

text - actual text represented by token

TokenBase

public TokenBase(TokenBase o)

Copy constructor

Parameters:

o - token to copy

Method Detail

getTypeString

protected abstract String getTypeString()

getLocalStart

public final int getLocalStart()

Returns:

Local start offset.

getLocalEnd

public final int getLocalEnd()

Returns:

Local end offset.

reuse

public final void reuse(int tokenType,
         int start,
         int end,
         int line,
         int column,
         CharSequence text)

lock

public void lock()

Locks this token. When locked, if this token is in a token pool, it will not be overwritten

isLocked

public boolean isLocked()

Returns whether this token can be overwritten when it is a member of a token pool

Returns:

true if we are locked

getText

public String getText()

Get the text represented by this token

Specified by:

getText in interface ICMToken

Overrides:

getText in class antlr.Token

Returns:

text represented by this token

See Also:

Token.getText()

getCharSequence

public CharSequence getCharSequence()

Returns the underlying CharSequence that represents the contents of this token

Returns:

a CharSequence or null

setText

public void setText(String text)

Set the text represented by this token

Overrides:

setText in class antlr.Token

Parameters:

text - text represented by this token

See Also:

Token.setText(java.lang.String)

setText

public void setText(CharSequence text)

Set the CharSequence represented by this token

Parameters:

text - text represented by this token

setLocation

public void setLocation(int start,
               int end,
               int line,
               int column)

getStart

public int getStart()

Description copied from interface: ICMToken

Returns the start offset of this token

Specified by:

getStart in interface ISourceLocation

Specified by:

getStart in interface ICMToken

Returns:

the start offset, or -1

setStart

public void setStart(int start)

getEnd

public int getEnd()

Description copied from interface: ICMToken

Returns the end offset of this token

Specified by:

getEnd in interface ISourceLocation

Specified by:

getEnd in interface ICMToken

Returns:

the end offset, or -1

setEnd

public void setEnd(int end)

getLine

public int getLine()

Description copied from interface: ICMToken

Returns the line of this token.. Line numbers start at 0, not 1.

Specified by:

getLine in interface ISourceLocation

Specified by:

getLine in interface ICMToken

Overrides:

getLine in class antlr.Token

Returns:

the line, or -1

setLine

public void setLine(int line)

Overrides:

setLine in class antlr.Token

matchesLine

public final boolean matchesLine(TokenBase other)

getColumn

public int getColumn()

Description copied from interface: ICMToken

Returns the column of this token. Column numbers staart at 0, not 1.

Specified by:

getColumn in interface ISourceLocation

Specified by:

getColumn in interface ICMToken

Overrides:

getColumn in class antlr.Token

Returns:

the column, or -1

setColumn

public void setColumn(int column)

Overrides:

setColumn in class antlr.Token

getEndLine

public int getEndLine()

Description copied from interface: ISourceLocation

Gets the local line number at the end. It is zero-based.

Specified by:

getEndLine in interface ISourceLocation

setEndLine

public void setEndLine(int line)

getEndColumn

public int getEndColumn()

Description copied from interface: ISourceLocation

Gets the local column number at the end. It is zero-based.

Specified by:

getEndColumn in interface ISourceLocation

setEndColumn

public void setEndColumn(int column)

isImplicit

public boolean isImplicit()

Determine whether or not this token is bogus (i.e. the start and end offsets are the same, which implies that it was inserted from an included file or during token fixup)

Specified by:

isImplicit in interface ICMToken

Returns:

true iff the token is bogus

toString

public String toString()

For debugging only. This format is nice in the Eclipse debugger.

Overrides:

toString in class antlr.Token

toDumpString

public String toDumpString()

For debugging only. This format is nice in a text file.

truncate

public void truncate(int trimLeft,
            int trimRight)

Reduce the span of the token by removing characters from the beginning and end. This is used to remove quote characters and such from tokens.

Parameters:

trimLeft - number of characters to remove from the left of the token

trimRight - number of characters to remove from the right of the token

adjustOffsets

public void adjustOffsets(int offsetAdjustment)

Adjust all associated offsets by the adjustment amount

Parameters:

offsetAdjustment - amount to add to offsets

adjustLocation

public void adjustLocation(int offsetAdjustment,
                  int lineAdjustment,
                  int columnAdjustment)

Adjust all associated offsets by the adjustment amount

Parameters:

offsetAdjustment - amount to add to offsets

lineAdjustment - amount to add to the line number

columnAdjustment - amount to add to the column number

storeLocalOffset

public final void storeLocalOffset()

Capture the current start/end offsets as this token's local offsets. This method is called in StreamingASTokenizer#nextTokenFromReader() after the token is initialized, and before being updated by IncludeHandler.onNextToken(org.apache.flex.compiler.internal.parsing.as.ASToken).

getSourcePath

public final String getSourcePath()

Description copied from interface: ICMToken

Get the source file path of the node. When the token is built from an included source, this method returns the file path of the included source. The return value can be null if the source file is not an included file, but not every non-included file has a null source path.

The source path will be copied to ISourceLocation.getSourcePath() during tree construction.

Specified by:

getSourcePath in interface ISourceLocation

Specified by:

getSourcePath in interface ICMToken

Returns:

source file path or null

setSourcePath

public final void setSourcePath(String path)

verify

public boolean verify()

Verifies that this token has its type and location information set.

This is used only in asserts.

getAbsoluteEnd

public int getAbsoluteEnd()

Description copied from interface: ISourceLocation

Gets the absolute starting offset. It is zero-based.

Specified by:

getAbsoluteEnd in interface ISourceLocation

getAbsoluteStart

public int getAbsoluteStart()

Description copied from interface: ISourceLocation

Gets the absolute starting offset. It is zero-based.

Specified by:

getAbsoluteStart in interface ISourceLocation