edu.stanford.nlp.process
Class LexerTokenizer
java.lang.Object
edu.stanford.nlp.process.AbstractTokenizer
edu.stanford.nlp.process.LexerTokenizer
- All Implemented Interfaces:
- Iterator, Tokenizer
- public class LexerTokenizer
- extends AbstractTokenizer
- implements Tokenizer
An implementation of Tokenizer
designed to work with
Lexer
implementing classes. Throw in a Lexer
on
construction and you get a Tokenizer
.
- Author:
- Roger Levy
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
LexerTokenizer
public LexerTokenizer(Lexer l)
LexerTokenizer
public LexerTokenizer(Lexer l,
Reader r)
getNext
protected Object getNext()
- Specified by:
getNext
in class AbstractTokenizer
main
public static void main(String[] args)
throws IOException
- for testing only
- Throws:
IOException
Stanford NLP Group