WaveMind: Towards a Conversational EEG Foundation Model Aligned to Textual and Visual Modalities
Year: 2025
Type: preprint
Abstract: Electroencephalography (EEG) interpretation using multimodal large language models (MLLMs) offers a novel approach for analyzing brain signals. However, the complex nature of brain activity introduces critical challenges: EEG signals simultaneously encode both cognitive processes and intrinsic neural states, creating a mismatch in EEG paired-data modality that hinders effective cross-modal representation learning. Through a pivot investigation, we uncover complementary relationships between these modalities. Leveraging this insight, we propose mapping EEG signals and their corresponding modalities into a unified semantic space to achieve generalized interpretation. To fully enable conversational capabilities, we further introduce WaveMind-Instruct-338k, the first cross-task EEG dataset for instruction tuning. The resulting model demonstrates robust classification accuracy while supporting flexible, open-ended conversations across four downstream tasks, thereby offering valuable insights for both neuroscience research and the development of general-purpose EEG models.
Source: arXiv (Cornell University)
Language: English
Subfield: Cognitive Neuroscience
Field: Neuroscience
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W4414804202
Cites:
Cited by:
Related to: