edu.stanford.nlp.io
Class AbstractStreamTokenizer
java.lang.Object
edu.stanford.nlp.io.AbstractStreamTokenizer
- All Implemented Interfaces:
- StreamTokenizer
- Direct Known Subclasses:
- NegraPennTokenizer, NegraTokenizer
- public abstract class AbstractStreamTokenizer
- extends Object
- implements StreamTokenizer
An abstract class for a tokenizer implementing
edu.stanford.nlp.io.StreamTokenizer, based on a Lexer
.
This class is intended to be extended by adding a constructor that
passes the appropriate Lexer to the constructor of this class. See
Lexer
on suggestions for constructing an appropriate
implementing class.
- Author:
- Roger Levy
Method Summary |
String |
next()
Returns the next token. |
void |
pushBack()
Pushes the previous token back. |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
AbstractStreamTokenizer
public AbstractStreamTokenizer(Lexer l)
next
public String next()
throws IOException
- Returns the next token. Satisfies the
edu.stanford.nlp.io.StreamTokenizer
interface.
- Specified by:
next
in interface StreamTokenizer
- Returns:
- the next token as a String. Returns null when end of
file has been reached.
- Throws:
IOException
pushBack
public void pushBack()
- Pushes the previous token back. Satisfies the
edu.stanford.nlp.io.StreamTokenizer
interface.
- Specified by:
pushBack
in interface StreamTokenizer
Stanford NLP Group