edu.stanford.nlp.process
Class WhitespaceTokenizer.WhitespaceTokenizerFactory<T extends HasWord>

java.lang.Object
  extended by edu.stanford.nlp.process.WhitespaceTokenizer.WhitespaceTokenizerFactory<T>
All Implemented Interfaces:
IteratorFromReaderFactory<T>, TokenizerFactory<T>
Enclosing class:
WhitespaceTokenizer<T extends HasWord>

public static class WhitespaceTokenizer.WhitespaceTokenizerFactory<T extends HasWord>
extends Object
implements TokenizerFactory<T>

A factory which vends WhitespaceTokenizers.

Author:
Christopher Manning

Constructor Summary
WhitespaceTokenizer.WhitespaceTokenizerFactory(LexedTokenFactory<T> factory)
           
WhitespaceTokenizer.WhitespaceTokenizerFactory(LexedTokenFactory<T> factory, boolean tokenizeNLs)
           
 
Method Summary
 Iterator<T> getIterator(Reader r)
          Return an iterator over the contents read from r.
 Tokenizer<T> getTokenizer(Reader r)
           
static TokenizerFactory<Word> newTokenizerFactory()
          Constructs a new TokenizerFactory that returns Word objects and treats carriage returns as normal whitespace.
 void setOptions(String options)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

WhitespaceTokenizer.WhitespaceTokenizerFactory

public WhitespaceTokenizer.WhitespaceTokenizerFactory(LexedTokenFactory<T> factory)

WhitespaceTokenizer.WhitespaceTokenizerFactory

public WhitespaceTokenizer.WhitespaceTokenizerFactory(LexedTokenFactory<T> factory,
                                                      boolean tokenizeNLs)
Method Detail

newTokenizerFactory

public static TokenizerFactory<Word> newTokenizerFactory()
Constructs a new TokenizerFactory that returns Word objects and treats carriage returns as normal whitespace. THIS METHOD IS INVOKED BY REFLECTION BY SOME OF THE JAVANLP CODE TO LOAD A TOKENIZER FACTORY. IT SHOULD BE PRESENT IN A TokenizerFactory.

Returns:
A TokenizerFactory that returns Word objects

getIterator

public Iterator<T> getIterator(Reader r)
Description copied from interface: IteratorFromReaderFactory
Return an iterator over the contents read from r.

Specified by:
getIterator in interface IteratorFromReaderFactory<T extends HasWord>
Parameters:
r - Where to read objects from
Returns:
An Iterator over the objects

getTokenizer

public Tokenizer<T> getTokenizer(Reader r)
Specified by:
getTokenizer in interface TokenizerFactory<T extends HasWord>

setOptions

public void setOptions(String options)
Specified by:
setOptions in interface TokenizerFactory<T extends HasWord>


Stanford NLP Group