Year: 2026
Type: conference-paper
Abstract: Modern speech and multimodal generation systems, such as singing voice conversion and audio-driven lip synchronization, critically depend on temporally stable and semantically unambiguous vocal representations.In practical pipelines, such representations are typically derived from music source separation (MSS) applied to mixed musical recordings.However, standard MSS paradigms often aggregate lead vocals and backing harmonies into a single vocal stream.Although multi-stem separation has been explored, existing approaches remain primarily optimized for signal-level reconstruction, often overlooking the intricate structural disentanglement required by downstream generation tasks.From a generation-oriented perspective, this motivates revisiting vocal separation from a representation learning standpoint.To this end, we propose VocalRep, a structureaware learning framework designed to disentangle lead vocals, harmonies, and instrument while enforcing role consistency across longform audio.By integrating global vocal identity conditioning with ranking-based objectives, Vo-calRep extracts role-consistent lead vocal representations without relying on explicit pitch or symbolic annotations.Experimental results demonstrate that VocalRep significantly improves performance in downstream singing voice conversion and audio-driven lip synchronization.
Language: English
Topic: Music and Audio Processing
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166875287