Work

Decoding Listened Speech From Raw EEG Signals Using Subject-Independent Embeddings and Pre-Trained Foundation Models

Year: 2026
Type: dissertation
Abstract: Decoding speech waveforms directly from non-invasive brain signals, such as Electroencephalography (EEG), is a popular research problem in Brain-Computer Interface (BCI).However, this task is inherently difficult due to two sources of variability, these are inter-subject and intra subject.In inter-subject the EEG patterns differ significantly across individuals due to differences in anatomy, cognitive style and electrode placement.Intra-subject variability has neural representations of the same spoken content varying depending upon context, prosody and cognitive state.Traditional approaches often rely on task-specific encoders trained on limited data, which leads to subject entanglement, it is a phenomenon where the model learns features unique to individual EEG recordings rather than generalizable speech-related patterns.Similarly, these models tend to overfit to the specific utterances seen during training, failing to generalize to unseen audio.Together, these two forms of variability create a major bottleneck for real-world BCI deployment, where both the speaker and the spoken content are unpredictable.This thesis utilizes a gradient reversal layer which is used as a subject discriminator to unlearn the subject related dependencies.Further, the EEG foundational models were integrated to provide generalization for unseen subjects data.The performance of these methods were evaluated using the Mel-Cepstral Distortion (MCD) metric and Melspectrogram Correlation (Mel-Corr) of the reconstructed speech waveform.MCD is an object metric used to evaluate the quality of synthesized speech by comparing it with the original speech.Mel-Corr is another metric used to measure the similarity between the mel-frequency of synthesized and original speech.Experimental results shows that the gradient reversal approach significantly reduces subject-dependent MCD by around 41 % leading to more stable reconstructions across a diverse pool of participants.Additionally, the integration of the foundation model yielded a 33 % reduction in MCD value, suggesting that transferred neural knowledge is critical for overcoming the limitations of small-scale datasets.Overall, this work contributes to a more generalized framework for neural speech decoding, moving the field closer to high quality, subject-independent communication interfaces.
Author: Kapoor, Aman
Language: English
Open Access status: green
OpenAlex ID: W7215219144
Cites: 128
Cited by:
Related to: 10