Increasing Deep Neural Network Acoustic Model Size for Large Vocabulary Continuous Speech Recognition
Year: 2014
Type: preprint
Abstract: Deep neural networks (DNNs) are now a central component of nearly all state-of-the-art speech recognition systems. Part of the promise of DNNs is their ability to represent increas-ingly complex functions as the number of DNN parameters increases. This paper investigates the performance of DNN-based hybrid speech recognition systems as DNN model size and training data increase. Using a distributed GPU architec-ture, we train DNN acoustic models roughly an order of mag-nitude larger than those typically found in speech recognition systems. DNNs of this scale achieve substantial reductions in final system word error rate despite training with a loss func-tion not tightly coupled to system error rate. However, training word error rate improvements do not translate to large improve-ments in test set word error rate for systems trained on the 300 hour Switchboard conversational speech corpus. Scaling DNN acoustic model size does prove beneficial on the Fisher 2,000 hour conversational speech corpus. Our results show that with sufficient training data, increasing DNN model size is an effec-tive, direct path to performance improvements. Moreover, even smaller DNNs benefit from a larger training corpus. Index Terms: speech recognition, neural networks, acoustic modeling
Source: arXiv (Cornell University)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W130515627