Year: 2026
Type: conference-paper
Abstract: We propose UniVocal, a unified framework that implicitly infers vocal modes from text context to pioneer Speech-Singing Code-Switching (SCS) Synthesis-a task where transitions are autonomously driven by textual semantics, akin to seamless human language blending.Unlike single-mode generation or systems relying on switching-control tags, our proposed UniVocal implicitly infers vocal modes solely from text context.To achieve this, we employ a data-efficient two-stage curriculum learning strategy that progressively trains a competitive TTS system to acquire the desired SCS capability.Addressing data scarcity, we introduce a scalable pipeline to synthesize diverse code-switching data that is both semantically and acoustically natural, alongside a new multi-scenario benchmark, SCSBench.To address limitations of semantic tokenizers in capturing acoustic details, we also introduce refined cent token and Chainof-Thought (CoT) generation for planning prosody before content generation, effectively enhancing empathetic speech generation and singing melody.Experimental results demonstrate that UniVocal achieves state-of-the-art performance on SCSBench while maintaining competitive performance on regular speech and singing tasks.
Institution: University of Science and Technology of China
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166814855