public static class ArabicTokenizer.ArabicTokenizerFactory<T extends HasWord> extends Object implements TokenizerFactory<T>, Serializable
Modifier and Type | Field and Description |
---|---|
protected LexedTokenFactory<T> |
factory |
protected Properties |
lexerProperties |
Modifier and Type | Method and Description |
---|---|
Iterator<T> |
getIterator(Reader r)
Return an iterator over the contents read from r.
|
Tokenizer<T> |
getTokenizer(Reader r) |
Tokenizer<T> |
getTokenizer(Reader r,
String extraOptions) |
static TokenizerFactory<CoreLabel> |
newTokenizerFactory() |
void |
setOptions(String options)
options: A comma-separated list of options
|
protected final LexedTokenFactory<T extends HasWord> factory
protected Properties lexerProperties
public static TokenizerFactory<CoreLabel> newTokenizerFactory()
public Iterator<T> getIterator(Reader r)
IteratorFromReaderFactory
getIterator
in interface IteratorFromReaderFactory<T extends HasWord>
r
- Where to read objects frompublic Tokenizer<T> getTokenizer(Reader r)
getTokenizer
in interface TokenizerFactory<T extends HasWord>
public void setOptions(String options)
setOptions
in interface TokenizerFactory<T extends HasWord>
public Tokenizer<T> getTokenizer(Reader r, String extraOptions)
getTokenizer
in interface TokenizerFactory<T extends HasWord>