Insight into Multiple References in an MT Evaluation Metric
Year: 2015
Type: conference-paper
Abstract: Current evaluation metrics in machine translation (MT) make poor use of multiple reference translations. In this paper we focus on the METEOR metric to gain in-depth insights into how best multiple references can be exploited. Results on five score selection strategies reveal that it is not always wise to choose the best (closest to MT) reference to generate the candidate score. We also propose two weighting approaches by taking into account the recurring information among references. The modified METEOR scores significantly increase the correlation with human judgments on accuracy and fluency evaluation at system level.
Authors: Ying Qin, Lucia Specia
Institutions: Beijing Foreign Studies University, University of Sheffield
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $5,392
OpenAlex ID: W2232659607
