Year: 2021
Type: preprint
Abstract: Modern neural machine translation (NMT) models have achieved competitive performance in standard benchmarks such as WMT. However, there still exist significant issues such as robustness, domain generalization, etc. In this paper, we study NMT models from the perspective of compositional generalization by building a benchmark dataset, CoGnition, consisting of 216k clean and consistent sentence pairs. We quantitatively analyze effects of various factors using compound translation error rate, then demonstrate that the NMT model fails badly on compositional generalization, although it performs remarkably well under traditional metrics.
Source: arXiv (Cornell University)
Authors: Li, Yafu, Yin, Yongjing, Chen, Yulong, Zhang, Yue
Institutions: Westlake University, Zhejiang University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Award: G8869030430
OpenAlex ID: W3167657000