Style and Prosody control for Zero-shot Speech Synthesis
Year: 2025
Type: conference-paper
Abstract: Modern text-to-speech (TTS) systems are often trained on vast datasets encompassing diverse speakers, styles, and conditions, resulting in a wide spectrum of potential outputs. While zero-shot synthesis leverages this learned variability for voice cloning, achieving fine-grained control remains an open area. This paper investigates simple, post-training techniques to manipulate prosody and style by directly operating on learned speech embeddings, specifically targeting zero-shot models that utilize a fixed-size, speech-derived embedding. We demonstrate how meaningful control directions within this latent space can be identified using reference variations from a single speaker. Vectors for prosody control (f0, duration, spectral tilt) are derived via regression, and style vectors via embedding arithmetic on distinct styles. This easily implementable method requires no model retraining and offers enhanced granular control for novel speakers, complementing base voice cloning.
DOI: 10.21437/ssw.2025-5
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7084142224