Year: 2018
Type: conference-paper
Abstract: This paper proposes a state-of-the-art recurrent neural network (RNN) language model that combines probability distributions computed not only from a final RNN layer but also from middle layers.Our proposed method raises the expressive power of a language model based on the matrix factorization interpretation of language modeling introduced by Yang et al. (2018).The proposed method improves the current state-of-the-art language model and achieves the best score on the Penn Treebank and WikiText-2, which are the standard benchmark datasets.Moreover, we indicate our proposed method contributes to two application tasks: machine translation and headline generation.
DOI: 10.18653/v1/d18-1489
Authors: Sho Takase, Jun Suzuki, Masaaki Nagata
Institutions: Tohoku University, NTT (Japan)
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W2888799392