This is a system for MT adequacy evaluation based on an RTE (Recognition of Textual Entailment) engine. It uses the NIST MetricsMATR 2008 XML input and output specifications (see the MetricsMATR evaluation plan).
S. Pado, M. Galley, D. Jurafsky, and C. Manning. 2009. Textual Entailment Features for Machine Translation Evaluation. Proceedings of WMT 2009.Note: the current download is essentially the Stanford submission to MetricsMATR 2008.
S. Pado, M. Galley, D. Jurafsky, and C. Manning. 2009. Robust Machine Translation Evaluation with Entailment Features. Proceedings of ACL 2009.
This system is not the full Stanford RTE system (which isn't currently distributed). It is not set up to work as an RTE system. It is only available as java archives (jar files), not as source code.
System requirements: The system requires Java 1.6. and 8 GB of memory.
For documentation, take a look at the included README and the paper.
We have 3 mailing lists for the MT evaluation system, all of which are shared
with other JavaNLP tools (with the exclusion of the parser). Each address is
java-nlp-userThis is the best list to post to in order to ask questions, make announcements, or for discussion among JavaNLP users. You have to subscribe to be able to use it. Join the list (via a webpage) or by emailing
email@example.com. (Leave the subject and message body empty.) You can also look at the list archives.
java-nlp-announceThis list will be used only to announce new versions of Stanford JavaNLP tools. So it will be very low volume (expect 1-3 message a year). Join the list by emailing
firstname.lastname@example.org. (Leave the subject and message body empty.)
java-nlp-supportThis list goes only to the software maintainers. It's a good address for licensing questions, etc. For general use and support questions, you're better off joining and using
java-nlp-user. You cannot join
java-nlp-support, but you can mail questions to
The system download is a 700 MB gzipped tar file (mainly consisting of included linguistic resources). When you unpack the tar file, you should have everything needed.