Difference between revisions of "More details on MERT options"

From NLPWiki
Jump to: navigation, search
(Created page with "=== OBJECTIVE === The following options are available: bleu ter terp: TER with synonym and paraphrase matching turned on (super slow). nist wer per bleu-ter")
 
(OBJECTIVE)
Line 1: Line 1:
 
=== OBJECTIVE ===
 
=== OBJECTIVE ===
 
The following options are available:
 
The following options are available:
   bleu
+
   "bleu" --  the percentage of common n-grams found in machine and reference translations (Papineni et al., 2002).
   ter
+
   "ter"  --  translation edit rate, i.e. shortest edit sequence to turn a machine translation into a reference (Snover et al., 2006).
   terp: TER with synonym and paraphrase matching turned on (super slow).
+
   "terp" --  a variant of TER with synonym and paraphrase matching turned on (super slow) (Snover et al., 2009).
   nist
+
   "nist" -- a variant of BLEU called the NIST which weights n-gram matches by how informative they are (Doddington, 2002).
   wer
+
   "wer"  -- word error rate (Nießen et al., 2000).
   per
+
   "per"
   bleu-ter
+
   "bleu-ter:w" -- linearly combine BLEU and TER with the weight w placed on TER, i.e. BLEU + w*TER. "bleu-ter" implies w=1.0.
 +
 
 +
For a comparison of these various metrics, see:
 +
@inproceedings{Cer:2010:BLM,
 +
  author = {Cer, Daniel and Manning, Christopher D. and Jurafsky, Daniel},
 +
  title = {The best lexical metric for phrase-based statistical MT system optimization},
 +
  booktitle = {Proceedings of NAACL},
 +
  year = {2010},
 +
}

Revision as of 16:45, 5 December 2013

OBJECTIVE

The following options are available:

 "bleu" --  the percentage of common n-grams found in machine and reference translations (Papineni et al., 2002).
 "ter"  --  translation edit rate, i.e. shortest edit sequence to turn a machine translation into a reference (Snover et al., 2006).
 "terp" --  a variant of TER with synonym and paraphrase matching turned on (super slow) (Snover et al., 2009).
 "nist" -- a variant of BLEU called the NIST which weights n-gram matches by how informative they are (Doddington, 2002).
 "wer"  -- word error rate (Nießen et al., 2000).
 "per"
 "bleu-ter:w" -- linearly combine BLEU and TER with the weight w placed on TER, i.e. BLEU + w*TER. "bleu-ter" implies w=1.0.

For a comparison of these various metrics, see:

@inproceedings{Cer:2010:BLM,
 author = {Cer, Daniel and Manning, Christopher D. and Jurafsky, Daniel},
 title = {The best lexical metric for phrase-based statistical MT system optimization},
 booktitle = {Proceedings of NAACL},
 year = {2010},
}