Year: 2025
Type: article
Abstract: Dysarthric speech recognition faces significant challenges of acoustic variability and data scarcity, and this study proposes a robust system by integrating generative adversarial network enhancement and large language model correction to address these issues effectively. The system employs three key components, including a multimodal recognition core that combines whisper‐medium encoder with LoRA‐fine‐tuned Llama‐3.1‐8B for end‐to‐end acoustic‐to‐semantic mapping, an improved CycleGAN module that generates synthetic dysarthric speech through Inception‐ResNet fusion blocks, and an intelligent error correction mechanism using N‐best hypothesis reranking with semantic constraints. Experiments on the UA‐Speech dataset show that the complete system achieves a 20.61% word error rate representing a 73.9% relative improvement over traditional end‐to‐end transformer automatic speech recognition. Under very low intelligibility conditions it maintains a 48.69% word error rate demonstrating robust recognition for severe pathological speech. Ablation studies validate each module's effectiveness, providing significant advances for dysarthric patient communication technologies.
Source: Advanced Intelligent Systems
Authors: Yibo He, Kah Phooi Seng, Chee Shen Lim, Li Minn Ang
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,970
Funder: University of the Sunshine Coast
OpenAlex ID: W4415558377