com.sun.tools.javac.parser
Class Scanner

java.lang.Object
  com.sun.tools.javac.parser.Scanner

All Implemented Interfaces:

Lexer

Direct Known Subclasses:

DocCommentScanner

public class Scanner
extends java.lang.Object
implements Lexer

The lexical analyzer maps an input stream consisting of ASCII characters and Unicode escapes into a token sequence.

This is NOT part of any API supported by Sun Microsystems. If you write code that depends on this, you do so at your own risk. This code and its internal interfaces are subject to change or deletion without notice.

Nested Class Summary

static class	Scanner.CommentStyle
static class	Scanner.Factory A factory for creating scanners.

Field Summary

private boolean	allowHexFloats Allow hex floating-point literals.
private int	bp
private char[]	buf The input buffer, index of next chacter to be read, index of one past last character in buffer.
private int	buflen
private char	ch The current character.
protected boolean	deprecatedFlag Has a @deprecated been encountered in last doc comment? this needs to be reset by client.
private int	endPos Character position just after the last character of the token.
private int	eofPos
private int	errPos The position where a lexical error occurred;
private static boolean	hexFloatsWork
private Keywords	keywords The keyword table.
private Log	log The log to be used for error reporting.
private Name	name The name of an identifier or token:
private Names	names The name table.
private int	pos The token's position, 0-based offset from beginning of text.
private int	prevEndPos The last character position of the previous token.
private int	radix The radix of a numeric literal token.
private char[]	sbuf A character buffer for literals.
private static boolean	scannerDebug
private int	sp
(package private) static boolean	surrogatesSupported Are surrogates supported?
private Token	token The token, set by nextToken().
private int	unicodeConversionBp The buffer index of the last converted unicode character

Constructor Summary

private	Scanner(Scanner.Factory fac) Common code for constructors.
protected	Scanner(Scanner.Factory fac, char[] input, int inputLength) Create a scanner from the input array.
protected	Scanner(Scanner.Factory fac, java.nio.CharBuffer buffer) Create a scanner from the input buffer.

Method Summary

private void	convertUnicode() Convert unicode escape; bp points to initial '\' character (Spec 3.3).
private void	dch() For debugging purposes: print character.
boolean	deprecatedFlag() Has a @deprecated been encountered in last doc comment? This needs to be reset by client with resetDeprecatedFlag.
private int	digit(int base) Convert an ASCII digit from its base (8, 10, or 16) to its value.
java.lang.String	docComment() Returns the documentation string of the current token.
int	endPos() Return the last character position of the current token.
int	errPos() Return the position where a lexical error occurred;
void	errPos(int pos) Set the position where a lexical error occurred;
Position.LineMap	getLineMap() Build a map for translating between line numbers and positions in the input.
char[]	getRawCharacters() Returns a copy of the input buffer, up to its inputLength.
char[]	getRawCharacters(int beginIndex, int endIndex) Returns a copy of a character array subset of the input buffer.
private static boolean	hexFloatsWork()
private boolean	isSpecial(char ch) Return true if ch can be part of an operator.
private void	lexError(int pos, java.lang.String key, java.lang.Object... args) Report an error at the given position using the provided arguments.
private void	lexError(java.lang.String key, java.lang.Object... args) Report an error at the current token position using the provided arguments.
Name	name() Return the name of an identifier or token for the current token.
void	nextToken() Read token.
int	pos() Return the current token's position: a 0-based offset from beginning of the raw input stream (before unicode translation)
int	prevEndPos() Return the last character position of the previous token.
protected void	processComment(Scanner.CommentStyle style) Called when a complete comment has been scanned.
protected void	processLineTerminator() Called when a line terminator has been processed.
protected void	processWhiteSpace() Called when a complete whitespace run has been scanned.
private void	putChar(char ch) Append a character to sbuf.
int	radix() Return the radix of a numeric literal token.
void	resetDeprecatedFlag()
private void	scanChar() Read next character.
private void	scanCommentChar() Read next character in comment, skipping over double '\' characters.
private void	scanDocComment() Scan a documention comment; determine if a deprecated tag is present.
private void	scanFraction() Read fractional part of floating point number.
private void	scanFractionAndSuffix() Read fractional part and 'd' or 'f' suffix of floating point number.
private void	scanHexExponentAndSuffix() Read fractional part of hexadecimal floating point number.
private void	scanHexFractionAndSuffix(boolean seendigit) Read fractional part and 'd' or 'f' suffix of floating point number.
private void	scanIdent() Read an identifier.
private void	scanLitChar() Read next character in character or string literal and copy into sbuf.
private void	scanNumber(int radix) Read a number.
private void	scanOperator() Read longest possible sequence of special characters and convert to token.
private char	scanSurrogates() Scan surrogate pairs.
java.lang.String	stringVal() The value of a literal token, recorded as a string.
private static boolean	surrogatesSupported()
Token	token() Return the current token, set by nextToken().
void	token(Token token) Sets the current token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

scannerDebug

private static boolean scannerDebug

token

private Token token

The token, set by nextToken().

allowHexFloats

private boolean allowHexFloats

Allow hex floating-point literals.

pos

private int pos

The token's position, 0-based offset from beginning of text.

endPos

private int endPos

Character position just after the last character of the token.

prevEndPos

private int prevEndPos

The last character position of the previous token.

errPos

private int errPos

The position where a lexical error occurred;

name

private Name name

The name of an identifier or token:

radix

private int radix

The radix of a numeric literal token.

deprecatedFlag

protected boolean deprecatedFlag

Has a @deprecated been encountered in last doc comment? this needs to be reset by client.

sbuf

private char[] sbuf

A character buffer for literals.

sp

private int sp

buf

private char[] buf

The input buffer, index of next chacter to be read, index of one past last character in buffer.

bp

private int bp

buflen

private int buflen

eofPos

private int eofPos

ch

private char ch

The current character.

unicodeConversionBp

private int unicodeConversionBp

The buffer index of the last converted unicode character

log

private final Log log

The log to be used for error reporting.

names

private final Names names

The name table.

keywords

private final Keywords keywords

The keyword table.

hexFloatsWork

private static final boolean hexFloatsWork

surrogatesSupported

static final boolean surrogatesSupported

Are surrogates supported?

Constructor Detail

Scanner

private Scanner(Scanner.Factory fac)

Common code for constructors.

Scanner

protected Scanner(Scanner.Factory fac,
                  java.nio.CharBuffer buffer)

Create a scanner from the input buffer. buffer must implement array() and compact(), and remaining() must be less than limit().

Scanner

protected Scanner(Scanner.Factory fac,
                  char[] input,
                  int inputLength)

Create a scanner from the input array. This method might modify the array. To avoid copying the input array, ensure that inputLength < input.length or input[input.length -1] is a white space character.

Parameters:

fac - the factory which created this Scanner

input - the input, might be modified

inputLength - the size of the input. Must be positive and less than or equal to input.length.

Method Detail

hexFloatsWork

private static boolean hexFloatsWork()

lexError

private void lexError(int pos,
                      java.lang.String key,
                      java.lang.Object... args)

Report an error at the given position using the provided arguments.

lexError

private void lexError(java.lang.String key,
                      java.lang.Object... args)

Report an error at the current token position using the provided arguments.

digit

private int digit(int base)

Convert an ASCII digit from its base (8, 10, or 16) to its value.

convertUnicode

private void convertUnicode()

Convert unicode escape; bp points to initial '\' character (Spec 3.3).

scanChar

private void scanChar()

Read next character.

scanCommentChar

private void scanCommentChar()

Read next character in comment, skipping over double '\' characters.

putChar

private void putChar(char ch)

Append a character to sbuf.

dch

private void dch()

For debugging purposes: print character.

scanLitChar

private void scanLitChar()

Read next character in character or string literal and copy into sbuf.

scanHexExponentAndSuffix

private void scanHexExponentAndSuffix()

Read fractional part of hexadecimal floating point number.

scanFraction

private void scanFraction()

Read fractional part of floating point number.

scanFractionAndSuffix

private void scanFractionAndSuffix()

Read fractional part and 'd' or 'f' suffix of floating point number.

scanHexFractionAndSuffix

private void scanHexFractionAndSuffix(boolean seendigit)

Read fractional part and 'd' or 'f' suffix of floating point number.

scanNumber

private void scanNumber(int radix)

Read a number.

Parameters:

radix - The radix of the number; one of 8, 10, 16.

scanIdent

private void scanIdent()

Read an identifier.

surrogatesSupported

private static boolean surrogatesSupported()

scanSurrogates

private char scanSurrogates()

Scan surrogate pairs. If 'ch' is a high surrogate and the next character is a low surrogate, then put the low surrogate in 'ch', and return the high surrogate. otherwise, just return 0.

isSpecial

private boolean isSpecial(char ch)

Return true if ch can be part of an operator.

scanOperator

private void scanOperator()

Read longest possible sequence of special characters and convert to token.

scanDocComment

private void scanDocComment()

Scan a documention comment; determine if a deprecated tag is present. Called once the initial /, * have been skipped, positioned at the second * (which is treated as the beginning of the first line). Stops positioned at the closing '/'.

stringVal

public java.lang.String stringVal()

The value of a literal token, recorded as a string. For integers, leading 0x and 'l' suffixes are suppressed.

Specified by:

stringVal in interface Lexer

nextToken

public void nextToken()

Read token.

Specified by:

nextToken in interface Lexer

token

public Token token()

Return the current token, set by nextToken().

Specified by:

token in interface Lexer

token

public void token(Token token)

Sets the current token.

Specified by:

token in interface Lexer

pos

public int pos()

Return the current token's position: a 0-based offset from beginning of the raw input stream (before unicode translation)

Specified by:

pos in interface Lexer

endPos

public int endPos()

Return the last character position of the current token.

Specified by:

endPos in interface Lexer

prevEndPos

public int prevEndPos()

Return the last character position of the previous token.

Specified by:

prevEndPos in interface Lexer

errPos

public int errPos()

Return the position where a lexical error occurred;

Specified by:

errPos in interface Lexer

errPos

public void errPos(int pos)

Set the position where a lexical error occurred;

Specified by:

errPos in interface Lexer

name

public Name name()

Return the name of an identifier or token for the current token.

Specified by:

name in interface Lexer

radix

public int radix()

Return the radix of a numeric literal token.

Specified by:

radix in interface Lexer

deprecatedFlag

public boolean deprecatedFlag()

Has a @deprecated been encountered in last doc comment? This needs to be reset by client with resetDeprecatedFlag.

Specified by:

deprecatedFlag in interface Lexer

resetDeprecatedFlag

public void resetDeprecatedFlag()

Specified by:

resetDeprecatedFlag in interface Lexer

docComment

public java.lang.String docComment()

Returns the documentation string of the current token.

Specified by:

docComment in interface Lexer

getRawCharacters

public char[] getRawCharacters()

Returns a copy of the input buffer, up to its inputLength. Unicode escape sequences are not translated.

Specified by:

getRawCharacters in interface Lexer

getRawCharacters

public char[] getRawCharacters(int beginIndex,
                               int endIndex)

Returns a copy of a character array subset of the input buffer. The returned array begins at the beginIndex and extends to the character at index endIndex - 1. Thus the length of the substring is endIndex-beginIndex. This behavior is like String.substring(beginIndex, endIndex). Unicode escape sequences are not translated.

Specified by:

getRawCharacters in interface Lexer

Parameters:

beginIndex - the beginning index, inclusive.

endIndex - the ending index, exclusive.

Throws:

IndexOutOfBounds - if either offset is outside of the array bounds

processComment

protected void processComment(Scanner.CommentStyle style)

Called when a complete comment has been scanned. pos and endPos will mark the comment boundary.

processWhiteSpace

protected void processWhiteSpace()

Called when a complete whitespace run has been scanned. pos and endPos will mark the whitespace boundary.

processLineTerminator

protected void processLineTerminator()

Called when a line terminator has been processed.

getLineMap

public Position.LineMap getLineMap()

Build a map for translating between line numbers and positions in the input.

Specified by:

getLineMap in interface Lexer

Returns:

a LineMap