Year: 2016
Type: conference-paper
Abstract: Neural network training has been shown to be advantageous in many natural language processing\napplications, such as language modelling or machine translation. In this paper, we describe in\ndetail a novel domain adaptation mechanism in neural network training. Instead of learning\nand adapting the neural network on millions of training sentences – which can be very timeconsuming or even infeasible in some cases – we design a domain adaptation gating mechanism\nwhich can be used in recurrent neural networks and quickly learn the out-of-domain knowledge\ndirectly from the word vector representations with little speed overhead. In our experiments,\nwe use the recurrent neural network language model (LM) as a case study. We show that the\nneural LM perplexity can be reduced by 7.395 and 12.011 using the proposed domain adaptation\nmechanism on the Penn Treebank and News data, respectively. Furthermore, we show that using\nthe domain-adapted neural LM to re-rank the statistical machine translation n-best list on the\nFrench-to-English language pair can significantly improve translation quality
Authors: Jian Zhang, Xiaofeng Wu, Andy Way, Qun Liu
Institution: Dublin City University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2573866231
