This page gives a couple of examples of building sentiment classifiers with the Stanford Classifier.
Short sentiment snippets (the Pang/Lee Rotten Tomatoes dataset)
This example shows good/bad ("fresh"/"rotten") sentiment classification based on a collection of short review excerpts from Rotten Tomatoes collected by Bo Pang and Lillian Lee. Begin by downloading the data from http://www.cs.cornell.edu/People/pabo/movie-review-data/.
You want the file http://www.cs.cornell.edu/People/pabo/movie-review-data/rt-polaritydata.tar.gz (it's referred to on the page as "sentence polarity dataset v1.0").
tar -xzf rt-polaritydata.tar.gz mv rt-polaritydata.README.1.0.txt rt-polaritydata cd rt-polaritydata ls
The data is in Windows CP1252, unfortunately. And, unfortunately, the Stanford Classifier doesn't yet support specifying a character encoding (unlike most of the rest of our packages). So, we'll convert it to utf-8.
iconv -f cp1252 -t utf-8 < rt-polarity.neg > rt-polarity.neg.utf8 iconv -f cp1252 -t utf-8 < rt-polarity.pos > rt-polarity.pos.utf8
perl -ne 'print "neg\t" . $_' < rt-polarity.neg.utf8 > rt-polarity.neg.utf8.tsv perl -ne 'print "pos\t" . $_' < rt-polarity.pos.utf8 > rt-polarity.pos.utf8.tsv
Here we'll divide off one tenth of the data to test one fold.
head -n 4798 < rt-polarity.neg.utf8.tsv > rt-polarity.utf8.train.tsv head -n 4798 < rt-polarity.pos.utf8.tsv >> rt-polarity.utf8.train.tsv tail -n 533 < rt-polarity.neg.utf8.tsv > rt-polarity.utf8.test.tsv tail -n 533 < rt-polarity.pos.utf8.tsv >> rt-polarity.utf8.test.tsv
Check it's all okay:
$ wc -l *.tsv 5331 rt-polarity.neg.utf8.tsv 5331 rt-polarity.pos.utf8.tsv 1066 rt-polarity.utf8.test.tsv 9596 rt-polarity.utf8.train.tsv 21324 total
Now classify it.
$ cat > rt1.prop trainFile = rt-polarity.utf8.train.tsv testFile = rt-polarity.utf8.test.tsv useClassFeature = true 1.splitWordsRegexp = \\s 1.useSplitWords = true useNB = true goldAnswerColumn = 0
java -cp ~/Software/stanford-classifier-2011-12-16/stanford-classifier.jar edu.stanford.nlp.classify.ColumnDataClassifier -prop rt1.prop
1066 examples in test set Cls neg: TP=431 FN=102 FP=107 TN=426; Acc 0.804 P 0.801 R 0.809 F1 0.805 Cls pos: TP=426 FN=107 FP=102 TN=431; Acc 0.804 P 0.807 R 0.799 F1 0.803 Micro-averaged accuracy/F1: 0.80394 Macro-averaged F1: 0.80394
java -cp ~/Software/stanford-classifier-2011-12-16/stanford-classifier.jar edu.stanford.nlp.classify.ColumnDataClassifier -prop rt2.prop
1066 examples in test set Cls neg: TP=446 FN=87 FP=110 TN=423; Acc 0.815 P 0.802 R 0.837 F1 0.819 Cls pos: TP=423 FN=110 FP=87 TN=446; Acc 0.815 P 0.829 R 0.794 F1 0.811 Micro-averaged accuracy/F1: 0.81520 Macro-averaged F1: 0.81511
Short sentiment snippets (the Kaggle competition version of the Stanford Sentiment Treebank)
This example is on the same Rotten Tomatoes data, but available in the forum of judgments on constituents of a parse of the examples, done initially for the Stanford Sentiment Dataset, but also distributed as a Kaggle competition.
If you download and unpack the data, you will have files train.tsv and test.tsv [really a devtest set, hopefully]. These both have a header row, which the Stanford Classifier doesn't by default know how to ignore, so you should edit the two files and delete the first row entirely.
The data is a tsv file with 4 columns: columns 0 and 1 are phrase and sentence ID and then columns 2 and 3 give a phrase and its sentiment score (from 0 through 4, ranging from negative to positive).
Since this data was already tokenized (with the Stanford Tokenizer), we can probably just use whitespace tokenization. (Unless maybe we wanted to try being clever like splitting on some hyphens.)
So, here's our first attempt at a prop file. We will do the initial experiments using 10-fold cross validation on the training data, since we don't have answers for the devtest data. This properties file gives us a Naive Bayes model:
$ cat krt1.prop trainFile = train.tsv crossValidationFolds = 10 testFile = test.tsv useClassFeature = true 2.splitWordsRegexp = \\s 2.useSplitWords = true useNB = true goldAnswerColumn = 3
$ java -cp "*" edu.stanford.nlp.classify.ColumnDataClassifier -prop krt1.prop