edu.stanford.nlp.international.arabic.process
Class ArabicTokenizer.ArabicTokenizerFactory<T extends HasWord>
java.lang.Object
edu.stanford.nlp.international.arabic.process.ArabicTokenizer.ArabicTokenizerFactory<T>
- All Implemented Interfaces:
- IteratorFromReaderFactory<T>, TokenizerFactory<T>
- Enclosing class:
- ArabicTokenizer<T extends HasWord>
public static class ArabicTokenizer.ArabicTokenizerFactory<T extends HasWord>
- extends Object
- implements TokenizerFactory<T>
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
factory
protected final LexedTokenFactory<T extends HasWord> factory
lexerProperties
protected Properties lexerProperties
newTokenizerFactory
public static TokenizerFactory<CoreLabel> newTokenizerFactory()
- Constructs a new TokenizerFactory that returns HasWord objects and
treats carriage returns as normal whitespace.
THIS METHOD IS INVOKED BY REFLECTION BY SOME OF THE JAVANLP
CODE TO LOAD A TOKENIZER FACTORY. IT SHOULD BE PRESENT IN A
TokenizerFactory.
- Returns:
- A TokenizerFactory that returns HasWord objects
getIterator
public Iterator<T> getIterator(Reader r)
- Description copied from interface:
IteratorFromReaderFactory
- Return an iterator over the contents read from r.
- Specified by:
getIterator
in interface IteratorFromReaderFactory<T extends HasWord>
- Parameters:
r
- Where to read objects from
- Returns:
- An Iterator over the objects
getTokenizer
public Tokenizer<T> getTokenizer(Reader r)
- Specified by:
getTokenizer
in interface TokenizerFactory<T extends HasWord>
setOptions
public void setOptions(String options)
- Specified by:
setOptions
in interface TokenizerFactory<T extends HasWord>
getTokenizer
public Tokenizer<T> getTokenizer(Reader r,
String extraOptions)
- Specified by:
getTokenizer
in interface TokenizerFactory<T extends HasWord>
Stanford NLP Group