Robust and Efficient Autoregressive Speech Synthesis with Dynamic Chunk-Wise Prediction Policy
Year: 2026
Type: conference-paper
Abstract: Recently, autoregressive (AR) language models have become a dominant approach in speech synthesis, offering scalable training and expressive generation. However, conventional AR models based on next-token prediction struggle with long speech sequences, often failing to build stable frame-to-frame attention. This increases latency and degrades quality, limiting real-time applicability. To address this, we propose DCAR, a dynamic chunk-wise autoregressive speech synthesis framework that enhances efficiency and robustness in intelligibility. DCAR is trained with multi-token prediction, enabling variable-span chunk prediction via a lightweight on-policy module. By dynamically adjusting prediction spans, it reduces sequence-length dependency while preserving high synthesis quality. Experiments show that DCAR outperforms traditional next-token prediction models, achieving up to 72.27% intelligibility improvement and 2.61× inference speedup. Additional analyses demonstrate its potential as a versatile foundation for next-generation speech synthesis.
Institution: Shanghai Jiao Tong University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: Research and Development
OpenAlex ID: W4416260826