Main program for testing SUTime
Processing a text string:
-in.type TEXT
-date YYYY-MM-dd
-i <text>
-o <output file>
Processing a text file:
-in.type TEXTFILE
-date YYYY-MM-dd
-i input.txt
-o <output file>
Running on Timebank
-in.type TIMEBANK_CSV
-i timebank.csv
-tempeval2.dct dct.txt
-o <output directory>
-eval <evaluation script>
Evaluating on Tempeval2
-in.type TEMPEVAL2
-i <directory with english data>
-o <output directory>
-eval <evaluation script>
-tempeval2.dct dct file (with document creation times)
TEMPEVAL2 (download from http://timeml.org/site/timebank/timebank.html)
Evaluation is token based.
TRAINING (english):
GUTIME:
precision 0.88
recall 0.71
f1-measure 0.79
accuracy 0.98
attribute type 0.92
attribute value 0.31 // LOW SCORE here is due to difference in format (no -,: in date)
After fixing some formats for GUTIME:
(GUTIME syntax is inconsistent at times (1991W 8WE, 19980212EV)
attribute value 0.67
SUTIME:
Default: sutime.teRelHeurLevel=NONE, restrictToTimex3=false
precision 0.873
recall 0.897
f1-measure 0.885
accuracy 0.991
P R F1
attribute type 0.918 | 0.751 0.802 0.776
attribute value 0.762 | 0.623 0.665 0.644
P R F1
mention attribute type 0.900 | 0.780 0.833 0.805
mention attribute value 0.742 | 0.643 0.687 0.664
sutime.teRelHeurLevel=MORE, restrictToTimex3=true
precision 0.876
recall 0.889
f1-measure 0.882
accuracy 0.991
P R F1
attribute type 0.918 | 0.744 0.798 0.770
attribute value 0.776 | 0.629 0.675 0.651
P R F1
mention attribute type 0.901 | 0.780 0.836 0.807
mention attribute value 0.750 | 0.649 0.696 0.672
------------------------------------------------------------------------------
TEST (english):
GUTIME:
precision 0.89
recall 0.79
f1-measure 0.84
accuracy 0.99
attribute type 0.95
attribute value 0.68
SUTIME:
Default: sutime.teRelHeurLevel=NONE, restrictToTimex3=false
precision 0.878
recall 0.963
f1-measure 0.918
accuracy 0.996
P R F1
attribute type 0.953 | 0.820 0.904 0.860
attribute value 0.791 | 0.680 0.750 0.713
P R F1
mention attribute type 0.954 | 0.837 0.923 0.878
mention attribute value 0.781 | 0.686 0.756 0.720
sutime.teRelHeurLevel=MORE, restrictToTimex3=true
precision 0.881
recall 0.963
f1-measure 0.920
accuracy 0.995
P R F1
attribute type 0.959 | 0.821 0.910 0.863
attribute value 0.818 | 0.699 0.776 0.736
P R F1
mention attribute type 0.961 | 0.844 0.936 0.888
mention attribute value 0.803 | 0.705 0.782 0.742