|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectedu.stanford.nlp.process.PTBTokenizer.PTBTokenizerFactory
public static class PTBTokenizer.PTBTokenizerFactory
Field Summary | |
---|---|
protected boolean |
tokenizeCRs
|
Constructor Summary | |
---|---|
PTBTokenizer.PTBTokenizerFactory()
Constructs a new PTBTokenizerFactory that treats carriage returns as normal whitespace. |
|
PTBTokenizer.PTBTokenizerFactory(boolean tokenizeCRs)
Constructs a new PTBTokenizer that optionally returns carriage returns as their own token. |
Method Summary | |
---|---|
java.util.Iterator |
getIterator(java.io.Reader r)
|
Tokenizer |
getTokenizer(java.io.Reader r)
|
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected boolean tokenizeCRs
Constructor Detail |
---|
public PTBTokenizer.PTBTokenizerFactory()
public PTBTokenizer.PTBTokenizerFactory(boolean tokenizeCRs)
PTBLexer.cr
.
Method Detail |
---|
public java.util.Iterator getIterator(java.io.Reader r)
getIterator
in interface IteratorFromReaderFactory
public Tokenizer getTokenizer(java.io.Reader r)
getTokenizer
in interface TokenizerFactory
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |