Work

Enhancing speech intelligibility in optical microphone systems through physics-informed data augmentation

Year: 2025

Type: article

Abstract: Laser doppler vibrometers (LDVs) facilitate noncontact speech acquisition; however, they are prone to material-dependent spectral distortions and speckle noise, which degrade intelligibility in noisy environments. This study proposes a data augmentation method that incorporates material-specific and impulse noises to simulate LDV-induced distortions. The proposed approach utilizes a gated convolutional neural network with HiFi-GAN to enhance speech intelligibility across various material and low signal-to-noise ratio (SNR) conditions, achieving a short-time objective intelligibility score of 0.76 at 0 dB SNR. These findings provide valuable insights into optimized augmentation and deep-learning techniques for enhancing LDV-based speech recordings in practical applications.

Source: JASA Express Letters

DOI: 10.1121/10.0036356

Authors: Jia-Wei Chen, Jia-Hui Li, Yi-Hao Jiang, Yi-Chang Wu, Ying-Hui Lai

Institutions: National Yang Ming Chiao Tung University, Ministry of Justice

Language: English

Topic: Speech and Audio Processing

Subfield: Signal Processing

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funder: National Science and Technology Council

Award: Hearing through Light: Enhancing Auditory Assistive Devices with Optical Microphones and Deep Learning Technology (Continuation Project)