|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
A Lexer interface to be used with AbstractStreamTokenizer. An easy way to build classes implementing this interface is with JFlex (http://jflex.de). Just make sure to include the following in the JFlex source file
In the Options and Macros section of the source file, include
%class JFlexDummyLexer
%standalone
%unicode
%int
%implements edu.stanford.nlp.io.Lexer
%{
public void pushBack(int n) {
yypushback(n);
}
public int getYYEOF() {
return YYEOF;
}
%}
Alternatively, you can customize your own lexer and get lots of flexibility out.
Field Summary | |
static int |
ACCEPT
|
static int |
IGNORE
|
Method Summary | |
int |
getYYEOF()
returns value for YYEOF |
void |
pushBack(int length)
Pushes back length character positions in the
lexer. |
int |
yylex()
Gets the next token from input and returns an integer value signalling what to do with the token. |
void |
yyreset(Reader r)
|
String |
yytext()
returns the matched input text region |
Field Detail |
public static final int ACCEPT
public static final int IGNORE
Method Detail |
public int yylex() throws IOException
IOException
public String yytext()
public void pushBack(int length)
length
character positions in the
lexer. Conventionally used to push back exactly one token.
public int getYYEOF()
public void yyreset(Reader r) throws IOException
IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |