edu.stanford.nlp.io
Interface Lexer


public interface Lexer

A Lexer interface to be used with AbstractStreamTokenizer. An easy way to build classes implementing this interface is with JFlex (http://jflex.de). Just make sure to include the following in the JFlex source file

In the Options and Macros section of the source file, include

%implements edu.stanford.nlp.io.Lexer

%{
public void pushBack(int n) {
yypushback(n);
}

public int getYYEOF() {
return YYEOF;
}
%}

Author:
Roger Levy

Field Summary
static int ACCEPT
           
static int IGNORE
           
 
Method Summary
 int getYYEOF()
          returns value for YYEOF
 void pushBack(int length)
          Pushes back length character positions in the lexer.
 int yylex()
          Gets the next token from input and returns an integer value signalling what to do with the token.
 String yytext()
          returns the matched input text region
 

Field Detail

ACCEPT

public static final int ACCEPT
See Also:
Constant Field Values

IGNORE

public static final int IGNORE
See Also:
Constant Field Values
Method Detail

yylex

public int yylex()
          throws IOException
Gets the next token from input and returns an integer value signalling what to do with the token.

IOException

yytext

public String yytext()
returns the matched input text region


pushBack

public void pushBack(int length)
Pushes back length character positions in the lexer. Conventionally used to push back exactly one token.


getYYEOF

public int getYYEOF()
returns value for YYEOF



Stanford NLP Group