Ambisonics domain Singing Voice Separation combining Deep Neural Network and Direction Aware Multichannel NMF
Year: 2021
Type: conference-paper
Abstract: This paper addresses the problem of multichannel source separation in ambisonics signals combining two powerful approaches, multichannel NMF with recent single-channel deep learning (DL) based spectrum inference. Individual source spectra are estimated from the zero-order (omnidirectional) spherical harmonic (SH) signal with a Masker-Denoiser twin network, able to model long-term temporal patterns of a musical piece. The initialized source spectrograms are used within a SHdomain spatial covariance mixing model based on multichannel non-negative matrix factorization (MNMF) that predicts the spatial characteristics of each source in order to refine the network prediction. The proposed framework is evaluated on the task of singing voice separation with a large dataset of simulated ambisonics signals using several setups. Experimental results show that our joint DL+SH-MNMF method outperforms both the individual monophonic DL-based separation, baseline multichannel MNMF and SH domain beamforming classical methods.
Institution: Universidad de Oviedo
Language: English
Topic: Speech and Audio Processing
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4235050345