Year: 2026
Type: preprint
Abstract: Spoken dialogue is a natural form of human--computer interaction, yet most speech language models remain limited to turn-based operation and lack real-time adaptability, such as user barge-in. Recent duplex speech-to-speech and speech-to-text models reduce latency by replacing multi-stage pipelines, but often compromise speech quality because accurate ASR, interruption handling, and high-fidelity synthesis must be optimized jointly. We propose VoiceChat-TTS, a low-latency, continuous, and streamable text-to-speech model for interactive agents. VoiceChat-TTS is driven directly by LLM text-token streams, supports explicit interruption via control tokens, and produces silence when no textual input is available. The model enables always-on, responsive speech generation while preserving modularity and high speech quality, and it supports mid-utterance interruptions without resetting the KV cache.
Source: arXiv (Cornell University)
Authors: Casanova, Edresson, Kim, Jaehyeon, Fuenmayor, Mariana Graterol, Hussain, Shehzeen, Klimkov, Viacheslav, Mendelev, Valentin, Desta, Mikyas, Neekhara, Paarth, Zelasko, Piotr, Chen, Chen, Rastorgueva, Elena, Hu, Ke, Pasad, Ankita, Yang, Xuesong, Alja'fari, Aya, Roy, Rajarshi, Badlani, Rohan, Roche, Jason, Li, Jason, Chen, Zhehuai
Institution: Nvidia (United States)
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7203614717
Cites:
Cited by:
Related to: