edu.stanford.nlp.parser.metrics
Class Evalb
java.lang.Object
edu.stanford.nlp.parser.metrics.AbstractEval
edu.stanford.nlp.parser.metrics.Evalb
- Direct Known Subclasses:
- Evalb.CBEval
public class Evalb
- extends AbstractEval
A Java re-implementation of the evalb bracket scoring metric (Collins, 1997) that accepts Unicode input.
"Collinization" should be performed on input trees prior to invoking the package programmatically.
"Collinization" refers to normalization of trees for things not counted in evaluation,
such as equivalencing PRT and ADVP, which has standardly been done in English evaluation.
A main method is provided that performs Collinization according to language specific settings.
This implementation was last validated against EVALB20080701 (http://nlp.cs.nyu.edu/evalb/)
by Spence Green on 22 Jan. 2010. Notwithstanding this, Sekine and collins' EVALB script has been
the common standard for constituency evaluation of parsers for the last decade. We always validate
any numbers we report with it, and we suggest that you do the same.
- Author:
- Dan Klein, Spence Green
Constructor Summary |
Evalb(java.lang.String str,
boolean runningAverages)
|
Evalb(java.lang.String str,
boolean runningAverages,
boolean charLevel)
|
Method Summary |
void |
evaluate(Tree guess,
Tree gold,
java.io.PrintWriter pw)
|
static void |
main(java.lang.String[] args)
Run the scoring metric on guess/gold input. |
protected java.util.Set<Constituent> |
makeObjects(Tree tree)
evalb only evaluates phrasal categories, thus constituents() does not
return objects for terminals and pre-terminals. |
Methods inherited from class edu.stanford.nlp.parser.metrics.AbstractEval |
display, display, evaluate, evaluate, getEvalbF1, getEvalbF1Percent, getExact, getExactPercent, getLastF1, getNum, getSentAveF1, precision |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Evalb
public Evalb(java.lang.String str,
boolean runningAverages)
Evalb
public Evalb(java.lang.String str,
boolean runningAverages,
boolean charLevel)
makeObjects
protected java.util.Set<Constituent> makeObjects(Tree tree)
- evalb only evaluates phrasal categories, thus constituents() does not
return objects for terminals and pre-terminals.
- Specified by:
makeObjects
in class AbstractEval
evaluate
public void evaluate(Tree guess,
Tree gold,
java.io.PrintWriter pw)
- Overrides:
evaluate
in class AbstractEval
main
public static void main(java.lang.String[] args)
- Run the scoring metric on guess/gold input. This method performs "Collinization."
The default language is English.
- Parameters:
args
- Usage: Evalb [-y|-l|lt|-v|-s|-c|-g|-e] goldFile guessFile
Stanford NLP Group