edu.stanford.nlp.io
Class AbstractStreamTokenizer

java.lang.Object
  |
  +--edu.stanford.nlp.io.AbstractStreamTokenizer
All Implemented Interfaces:
StreamTokenizer

public abstract class AbstractStreamTokenizer
extends Object
implements StreamTokenizer

An abstract class for a tokenizer implementing edu.stanford.nlp.io.StreamTokenizer, based on a Lexer.

This class is intended to be extended by adding a constructor that passes the appropriate Lexer to the constructor of this class. See Lexer on suggestions for constructing an appropriate implementing class.

Author:
Roger Levy

Constructor Summary
AbstractStreamTokenizer(Lexer l)
           
 
Method Summary
 String next()
          Returns the next token.
 void pushBack()
          Pushes the previous token back.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

AbstractStreamTokenizer

public AbstractStreamTokenizer(Lexer l)
Method Detail

next

public String next()
            throws IOException
Returns the next token. Satisfies the edu.stanford.nlp.io.StreamTokenizer interface.

Specified by:
next in interface StreamTokenizer
Returns:
the next token as a String. Returns null when end of file has been reached.
Throws:
IOException

pushBack

public void pushBack()
Pushes the previous token back. Satisfies the edu.stanford.nlp.io.StreamTokenizer interface.

Specified by:
pushBack in interface StreamTokenizer


Stanford NLP Group