org.apache.flex.compiler.internal.parsing.mxml

Class BaseRawMXMLTokenizer

java.lang.Object

org.apache.flex.compiler.internal.parsing.mxml.BaseRawMXMLTokenizer

public abstract class BaseRawMXMLTokenizer
extends Object

Base class for JFlex-based tokenizers (RawScriptTokenizer, RawTagTokenizer, RawCSSTokenizer, RawMetadataTokenizer). Tokenizers should use %extends BaseRawTokenizer and provide a definitino for getColumn(). yytext()) will be generated automatically.

Field Summary

Fields

Modifier and Type	Field and Description
protected StringBuilder	aggregateContents Contents of aggregate
protected int	aggregateStart Start offset of aggregate
protected int	aggregateStartColumn Column number of aggregate
protected int	aggregateStartLine Line number of aggregate
protected List<ICompilerProblem>	problems

Constructor Summary

Constructors

Constructor and Description
BaseRawMXMLTokenizer()

Method Summary

Methods

Modifier and Type	Method and Description
protected antlr.Token	buildAggregateToken(int type) Build a token from the current aggregated text and the given type
protected antlr.Token	buildToken(int type) Build a token of the specified type, using the current yytext(), getOffset(), getLine(), and getColumn().
protected antlr.Token	buildToken(int type, int start, int end, int line, int column, String text)
protected antlr.Token	buildToken(int type, String text) Builds a token with the specified type and text, using the current getOffset(), getLine(), and getColumn().
protected void	continueAggregate() Add the current yytext() to the current aggregate
protected void	continueAggregate(antlr.Token token)
protected void	fillBuffer(StringBuilder builder)
ASToken	getBufferToken()
abstract int	getColumn() Gets the current column number of the tokenizer.
protected String	getContext(int line) Get the current context as a string (to help with debugging)
String	getLastTokenText()
int	getLastTokenType()
abstract int	getLine() Gets the current line number of the tokenizer.
abstract int	getOffset() Gets the current offset of the tokenizer into the file buffer.
List<ICompilerProblem>	getProblems()
protected boolean	hasAggregateContents()
boolean	hasBufferToken()
boolean	hasProblems()
boolean	isInE4XDatabinding()
void	reset()
protected void	setLastToken(antlr.Token token)
protected void	startAggregate() Initialize a new aggregate with the current yytext() and position
protected void	startAggregate(antlr.Token token)
abstract String	yytext() Gets the text of the current token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

aggregateStart

protected int aggregateStart

Start offset of aggregate

aggregateStartLine

protected int aggregateStartLine

Line number of aggregate

aggregateStartColumn

protected int aggregateStartColumn

Column number of aggregate

aggregateContents

protected StringBuilder aggregateContents

Contents of aggregate

problems

protected List<ICompilerProblem> problems

Constructor Detail

BaseRawMXMLTokenizer

public BaseRawMXMLTokenizer()

Method Detail

yytext

public abstract String yytext()

Gets the text of the current token. This is implemented by JFlex.

Returns:

The text of current token

getOffset

public abstract int getOffset()

Gets the current offset of the tokenizer into the file buffer. This should be implemented in the grammar file by returning JFlex's zzchar.

Returns:

The current offset.

getLine

public abstract int getLine()

Gets the current line number of the tokenizer. Line numbers start at 1, not 0. This should be implemented in the grammar file by returning JFlex's zzline.

Returns:

The current line.

getColumn

public abstract int getColumn()

Gets the current column number of the tokenizer. Column numbers start at 1, not 0. This should be implemented in the grammar file by returning JFlex's zzcolumn.

Returns:

The current column.

buildToken

protected antlr.Token buildToken(int type,
                     int start,
                     int end,
                     int line,
                     int column,
                     String text)

buildToken

protected antlr.Token buildToken(int type,
                     String text)

Builds a token with the specified type and text, using the current getOffset(), getLine(), and getColumn(). The grammar file should not override this.

Parameters:

type - token type (based on the appropriate XxxTokenTypes interface)

Returns:

new token

buildToken

protected antlr.Token buildToken(int type)

Build a token of the specified type, using the current yytext(), getOffset(), getLine(), and getColumn(). The grammar file cannot override this.

Parameters:

type - token type (based on the appropriate XxxTokenTypes interface)

Returns:

new token

reset

public void reset()

getLastTokenType

public final int getLastTokenType()

getLastTokenText

public final String getLastTokenText()

setLastToken

protected void setLastToken(antlr.Token token)

startAggregate

protected final void startAggregate()

Initialize a new aggregate with the current yytext() and position

startAggregate

protected final void startAggregate(antlr.Token token)

continueAggregate

protected void continueAggregate()

Add the current yytext() to the current aggregate

continueAggregate

protected final void continueAggregate(antlr.Token token)

fillBuffer

protected void fillBuffer(StringBuilder builder)

hasAggregateContents

protected final boolean hasAggregateContents()

buildAggregateToken

protected antlr.Token buildAggregateToken(int type)

Build a token from the current aggregated text and the given type

Parameters:

type - token type (based on the appropriate XxxTokenTypes interface)

Returns:

new token

getContext

protected String getContext(int line)

Get the current context as a string (to help with debugging)

Parameters:

line - current line number

Returns:

description of current context

hasProblems

public boolean hasProblems()

Returns:

true if we encountered errors while tokenizing

getProblems

public List<ICompilerProblem> getProblems()

Returns:

any problems we encountered while parsing

hasBufferToken

public boolean hasBufferToken()

getBufferToken

public ASToken getBufferToken()

isInE4XDatabinding

public boolean isInE4XDatabinding()