Year: 2025
Type: article
Abstract: A robust multichannel speaker diarization and separation system is proposed by exploiting the spatiotemporal activity of the speakers. The system is realized in a hybrid architecture that combines the array signal processing units and the deep learning units. For speaker diarization, a spatial coherence matrix across time frames is computed based on the whitened Relative Transfer Functions of the microphone array. This serves as a robust feature for subsequent machine learning without the need for prior knowledge of the array configuration. A computationally efficient modified End-to-End Neural Diarization system in the Encoder-Decoder-based Attractor network is constructed to estimate the speaker activity from the spatial coherence matrix. For speaker separation, we propose the Global and Local Activity-driven Speaker Extraction network to separate speaker signals via speaker-specific global and local spatial activity functions. The local spatial activity functions depend on the coherence between the whitened Relative Transfer Functions of each time-frequency bin and the target speaker-dominant bins. The global spatial activity functions are computed from the global spatial coherence functions based on frequency-averaged local spatial activity functions. Experimental results have demonstrated superior speaker, diarization, counting, and separation performance achieved by the proposed system with low computational complexity compared to the pre-selected baselines.
DOI: 10.1121/10.0035830
Authors: Yicheng Hsu, Ssuhan Chen, Yuhsin Lai, Chingyen Wang, Mingsian R. Bai
Institution: National Tsing Hua University
Language: English
Topic: Speech and Audio Processing
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4407526694