edu.stanford.nlp.trees.international.arabic
Class ArabicTreebankTokenizer
java.lang.Object
edu.stanford.nlp.process.AbstractTokenizer<String>
edu.stanford.nlp.process.TokenizerAdapter
edu.stanford.nlp.trees.PennTreebankTokenizer
edu.stanford.nlp.trees.international.arabic.ArabicTreebankTokenizer
- All Implemented Interfaces:
- Tokenizer<String>, Iterator<String>
public class ArabicTreebankTokenizer
- extends PennTreebankTokenizer
Builds a tokenizer for English PennTreebank (release 2) trees.
This is currently internally implemented via a java.io.StreamTokenizer.
- Author:
- Christopher Manning
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
ArabicTreebankTokenizer
public ArabicTreebankTokenizer(Reader r)
getNext
public String getNext()
- Internally fetches the next token.
- Overrides:
getNext
in class TokenizerAdapter
- Returns:
- the next token in the token stream, or null if none exists.
main
public static void main(String[] args)
throws IOException
- Throws:
IOException
Stanford NLP Group