Neural decoding of speech using deep neural ensembles
Year: 2026
Type: preprint
Abstract: Speech brain-computer interfaces (BCIs) can restore rapid communication to people with paralysis, but decoding errors still limit performance. In recent brain-to-text decoding competitions, deep ensemble methods, which combine predictions from multiple independently trained decoders, have delivered striking accuracy improvements and account for the largest gains over baseline approaches. However, these methods have not previously been tested in real-time, require substantial computational resources, and their performance under various clinically relevant constraints remains poorly understood. Here, we present the first closed-loop test of deep ensembles in a participant with bilateral intracortical microelectrode arrays, demonstrating a reduction in word error rate from 33.7% to 26.0% on a large-vocabulary task. Using additional data from three participants, we then assess how these gains depend on baseline error rate, training dataset size, and ensemble size, including the resource-accuracy tradeoffs most relevant for real-world deployment. Finally, we introduce a computationally efficient pseudoensembling approach based on test-time augmentation that improves decoding accuracy while requiring only a single base decoder, greatly reducing the computational burden of ensembling. Together, these results show that the benefits of deep ensembling can be realized in real time and under practical resource constraints, bringing speech BCIs closer to broader clinical adoption.
Authors: Seonghyun Yoon, Donald T. Avansino, Sasidhar Madugula, Alisa D. Levin, Chaofei Fan, Benyamin Abramovich Krasa, Akansha Singh, Christina Vo, Nick V. Hahn, Nicholas S. Card, Zachery Fogg, Maitreyee Wairagkar, Samuel R. Nason-Tomaszewski, Brandon G. Jacques, Payton H. Bechefsky, Carrina Iacobacci, Darrel R. Deo, Leigh R. Hochberg, David M. Brandman, Sergey D. Stavisky, Nicholas Au Yong, Chethan Pandarinath, Jaimie M. Henderson, Francis R. Willett
Language: English
Subfield: Cognitive Neuroscience
Field: Neuroscience
Domain: Life Sciences
Open Access status: green
Funders: Howard Hughes Medical Institute, U.S. Department of Veterans Affairs, Burroughs Wellcome Fund, Simons Foundation, A.P. Giannini Foundation, Searle Scholars Program, National Institutes of Health, Wu Tsai Neurosciences Institute, Stanford University, National Institute on Deafness and Other Communication Disorders, National Institute of Neurological Disorders and Stroke, Eunice Kennedy Shriver National Institute of Child Health and Human Development
Awards: G439081936, G4462149011, Fusing motor neuroscience and artificial intelligence to create next-generation neural prostheses., G844067152, Restoring Dexterous Hand Function with Artificial Neural Network-Based Brain-Computer Interfaces, Engaging new cognitive and motor signals to improve communication prostheses, Understanding and restoring speech production using an intracortical brain-computer interface, G993666374
OpenAlex ID: W7163582849