TER-Plus: TERp

Translation-Edit-Rate plus pronounced "terp"
Matthew Snover, Nitin Madnani. Bonnie Dorr and Rich Schwartz.

About

TERp is an automatic evaluation metric for Machine Translation, which takes as input a set of reference translations, and a set of machine translation output for that same data. It aligns the MT output to the reference translations, and measures the number of 'edits' needed to transform the MT output into the reference translation. TERp is an extension of TER (Translation Edit Rate) that utilizes phrasal substitutions (using automatically generated paraphrases), stemming, synonyms, relaxed shifting constraints and other improvements.


Citations

Citation and description for TER-Plus:

      Matthew Snover, Nitin Madnani, Bonnie Dorr, and Richard Schwartz, "Fluency, Adequacy, or HTER? Exploring Different Human Judgments with a Tunable MT Metric", Proceedings of the Fourth Workshop on Statistical Machine Translation at the 12th Meeting of the European Chapter of the Association for Computational Linguistics (EACL-2009), Athens, Greece, March, 2009.
[PDF]

Citation and description for TER:

      Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul, "A Study of Translation Edit Rate with Targeted Human Annotation," Proceedings of Association for Machine Translation in the Americas, 2006.


Software and Data Downloads

Version 0.1 (current version) 03/16/09 Improves TER with the addition of synonyms, stem matching, paraphrases, shift constraints, and optimized edit costs. Includes phrase table filterer, and edit cost optimization. [Documentation]
Software Download (without paraphrase table) Download [terp.v1.tgz] (6.3 MB)
Paraphrase PhraseTable Only Download [terp-pt.v1.tgz] (122 MB)


Matthew G Snover