Year: 2020
Type: conference-paper
Abstract: This paper presents two strong methods, CTC and Imputer, for non-autoregressive machine translation that model latent alignments with dynamic programming.We revisit CTC for machine translation and demonstrate that a simple CTC model can achieve state-of-theart for single-step non-autoregressive machine translation, contrary to what prior work indicates.In addition, we adapt the Imputer model for non-autoregressive machine translation and demonstrate that Imputer with just 4 generation steps can match the performance of an autoregressive Transformer baseline.Our latent alignment models are simpler than many existing non-autoregressive translation baselines; for example, we do not require target length prediction or re-scoring with an autoregressive model.On the competitive WMT'14 En→De task, our CTC model achieves 25.7 BLEU with a single generation step, while Imputer achieves 27.5 BLEU with 2 generation steps, and 28.0 BLEU with 4 generation steps.This compares favourably to the autoregressive Transformer baseline at 27.8 BLEU.
Authors: Chitwan Saharia, William Chan, Saurabh Saxena, Mohammad Norouzi
Institution: Google (United States)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W3016440711