More details on MERT options

From NLPWiki
Jump to: navigation, search

OBJECTIVE

The following options are available:

 "bleu" --  the percentage of common n-grams found in machine and reference translations (Papineni et al., 2002).
 "ter"  --  translation edit rate, i.e. shortest edit sequence to turn a machine translation into a reference (Snover et al., 2006).
 "terp" --  a variant of TER with synonym and paraphrase matching turned on (super slow) (Snover et al., 2009).
 "nist" --  a variant of BLEU which weights n-gram matches by how informative they are (Doddington, 2002).
 "wer"  --  word error rate (Nießen et al., 2000).
 "per"
 "bleu-ter:w" -- linearly combine BLEU and TER with the weight w placed on TER, i.e. BLEU + w*TER. "bleu-ter" implies w=1.0.

For a comparison of these various metrics, see:

@inproceedings{Cer:2010:BLM,
 author = {Cer, Daniel and Manning, Christopher D. and Jurafsky, Daniel},
 title = {The best lexical metric for phrase-based statistical MT system optimization},
 booktitle = {Proceedings of NAACL},
 year = {2010},
}