Year: 2024
Type: conference-paper
Abstract: In recent years, Non-Autoregressive Translation (NAT) has received lots of attention because of its outstanding decoding speed. However, there is a certain gap between the NAT model and its autoregressive comparator. It, how to effectively narrow this gap, remains to be further investigated. In this work, we use BERT to improve the performance of NAT model. Specifically, we add a CTC model on the top of BERT for extracting semantic and learning alignment. In order to test the effect of the model, we conducted the experiments on three public translation datasets. The experimental results show that our model can significantly outperforms existing non-autoregressive baselines.
Authors: Zijian Fu, Shuheng Wang
Institution: Nanyang Institute of Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Funder: Nanyang Institute of Technology
Award: G3551842816
OpenAlex ID: W4409965471