Work

Finding the Optimal Vocabulary Size for Neural Machine Translation

Year: 2020

Type: conference-paper

Abstract: We cast neural machine translation (NMT) as a classification task in an autoregressive setting and analyze the limitations of both classification and autoregression components.Classifiers are known to perform better with balanced class distributions during training.Since the Zipfian nature of languages causes imbalanced classes, we explore its effect on NMT.We analyze the effect of various vocabulary sizes on NMT performance on multiple languages with many data sizes, and reveal an explanation for why certain vocabulary sizes are better than others. 1

DOI: 10.18653/v1/2020.findings-emnlp.352

Authors: Thamme Gowda, Jonathan May

Institution: University of Southern California

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold