Work

Reusing Weights in Subword-Aware Neural Language Models

Year: 2018

Type: conference-paper

Abstract: The authors introduce methods for reusing subword embeddings and other parameters in subword-aware neural language models. Techniques improve syllable- and morpheme-aware models' performance while greatly reducing model size. A practical principle is identified: when reusing embedding layers at the output, they should be tied consecutively from bottom up. The best morpheme-aware model significantly outperforms word-level baselines across languages with 20–87 % fewer parameters.

Source: Nazarbayev University Repository (Nazarbayev University)

DOI: 10.18653/v1/n18‑1128

Authors: Takhanov, Rustem, Assylbekov, Zhenisbek, Assylbekov, Zhenisbek

Institution: Nazarbayev University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Nvidia, Ministry of Education and Science of the Republic of Kazakhstan

Award: G4689944890