Work

Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation

Year: 2018

Type: conference-paper

Abstract: Beam search is widely used in neural machine translation, and usually improves translation quality compared to greedy search.It has been widely observed that, however, beam sizes larger than 5 hurt translation quality.We explain why this happens, and propose several methods to address this problem.Furthermore, we discuss the optimal stopping criteria for these methods.Results show that our hyperparameter-free methods outperform the widely-used hyperparameter-free heuristic of length normalization by +2.0 BLEU, and achieve the best results among all methods on Chinese-to-English translation.

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/v1/d18-1342

Authors: Yilin Yang, Liang Huang, Mingbo Ma

Institutions: Oregon State University, Baidu (China)

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Science Foundation, Defense Advanced Research Projects Agency

Awards: G1037014486, RI: Small: Fast and Accurate Natural Language Parsing and Generation by Marrying Deep Learning with Dynamic Programming, G3959814348, EAGER: Collaborative Research: Scaling Up Discriminative Learning for Natural Language Understanding and Translation