Get started
Work

Het aanpakken van dataschaarste en -variabiliteit voor dysartrische spraakherkenning en -begrip

Year: 2025
Type: dissertation
Abstract: Dysarthria is a speech disorder caused by neurological damage that disrupts the speech motor system, resulting in poor pronunciations and significant communication challenges. Advances in speech technologies, particularly Automatic Speech Recognition (ASR) and Spoken Language Understanding (SLU), have proven beneficial for dysarthric speech processing. These technologies, powered by artificial neural networks (ANNs), show great promise in enhancing task performance for individuals with speech impairments. However, one of the primary challenges in this field is the scarcity of dysarthric speech data. Collecting such data is difficult, as dysarthric speakers often face challenges in producing lengthy speech samples. Moreover, variability in dysarthric speech patterns, driven by factors such as the type and progression of the underlying condition, adds complexity, leading to models trained on one group of dysarthric speakers performing poorly when tested on others. An effective ASR or SLU system for dysarthric speech must go beyond achieving good performance. It should also generalize well to new users who were not included in the training data, a setting referred to as speaker-independent recognition. This would reduce the need for new users to contribute significant amounts of training data, making the system more practical and minimizing users' effort to interact with the system. Additionally, in low-resource scenarios, where only a small amount of target speaker data is available, the model must maximize the utility of this limited data. Both speaker-independent and low-resource settings require the model to overcome the challenges of data scarcity and variability across dysarthric speakers. The overarching goal of this research is to enhance model performance for dysarthric speech, focusing on both of these crucial scenarios. To achieve these objectives, this study explores two key methodologies. The first focuses on representation learning, where the model extracts dysarthria-invariant features that can generalize across various speakers. This is achieved by disentangling speech features into dysarthria-specific and dysarthria-invariant components, allowing models trained on existing canonical or dysarthric speech to perform better on unseen dysarthric speakers. The second methodology involves modeling speech characteristics. In low-resource settings, the speech characteristics of target dysarthric speakers are modeled using their limited data efficiently, enabling personalized model adaptation. In speaker-independent settings, the characteristics of existing canonical and dysarthric speech recordings are leveraged to optimize knowledge transfer, enhancing the model's ability in target speech recognition. The study begins by addressing feature disentanglement through the use of the Factorized Hierarchical Variational Autoencoder (FHVAE) model. This model separates sequence-related (speaker) information from segment-related (content) information. To ensure accurate dysarthria-related feature extraction, regularization is applied to the latent variables based on segment-level labels. The success of this approach is demonstrated through a dysarthria type classification task, confirming that the extracted variables indeed capture disorder-related information. Moreover, speaker-related variables outperform traditional filter bank features and the original FHVAE model. Since obtaining segment-level phone labels in dysarthric speech datasets is challenging, the study then avoids relying on these labels and instead enhances feature disentanglement by integrating adversarial training into the FHVAE model. This approach allows for effective disentanglement with reduced supervision. By using both dysarthric and canonical speech as inputs, the model treats the content variable encoder as a generator and incorporates a discriminator to differentiate between the two speech types. This setup enables the model to learn dysarthria-invariant speech features. Evaluations on SLU tasks confirm the dysarthria-invariance of the content variable, while speech-type classification tasks demonstrate the model's ability to effectively separate speaker and content information. The research then shifts focus to methodologies that model speech characteristics and addresses low-resource settings through the use of parameter-efficient techniques, including Adapter modules and Adapter Fusion. These methods optimize the utilization of limited data from target dysarthric speakers, allowing for personalized model adaptation. Remarkably, these techniques yield recognition performance comparable to traditional fine-tuning when sufficient training data is available and even surpass fine-tuning in cases with minimal data. Further refinements to the Adapter Fusion method, such as reducing the layer size and reconfiguring the value linear layer weight matrix using Householder transformations, maintain comparable recognition performance while significantly reducing the number of trainable parameters. Finally, the study examines the speaker-independent setting, exploring training strategies that optimize knowledge transfer from existing canonical and dysarthric speech data to target dysarthric speech. The research also addresses vocabulary-mismatch scenarios, where the training vocabulary differs from the test vocabulary. For speaker-independent settings, all available source data is used to determine the best strategy for generalizing to new speakers. In the vocabulary-mismatch case, the model's performance limitations are highlighted. The introduction of phonological features as intermediate representations for phone recognition allows leveraging prior phonetic knowledge. Experimental results demonstrate that phonological features significantly improve the model's recognition performance in both speaker-independent and vocabulary-mismatched settings.
Source: Lirias
Author: Qi, Jinzi
Language: English
Open Access status: green
OpenAlex ID: W7113208202
Cites:
Cited by:
Related to: