Work

Contrastive Learning with Variational Regularization for Multi-Session EEG-to-Speech Decoding

Year: 2026

Type: preprint

Abstract: Reconstructing heard speech from non-invasive electroencephalography (EEG) is challenging due to a low signal-to-noise ratio (SNR) and inter-session variability. While trial averaging improves the SNR, it is difficult to apply to continuous speech. We instead use repeated EEG responses to the same stimulus across different sessions as positive pairs for contrastive learning, and introduce variational regularization that, combined with this contrastive objective, keeps the encoder representation space broad. Experiments on a Japanese EEG dataset show that combining the session-invariant strategy with variational regularization improves the character error rate (CER) while maintaining mel-spectrogram reconstruction fidelity. Session probing confirms that the encoder representations achieve session-invariance.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2608.16360

Authors: Mizuno, Tomoaki, Nakashika, Toru

Institution: University of Electro-Communications

Topic: EEG and Brain-Computer Interfaces

Subfield: Cognitive Neuroscience

Field: Neuroscience

Domain: Life Sciences

Open Access status: green

Funder: Japan Society for the Promotion of Science

Award: Establishment of next-generation communication using non-invasive EEG-based linguistic and non-linguistic speech synthesis

OpenAlex ID: W7203682029

Cites:

Cited by:

Related to: