Work

The challenge of out-of-distribution detection in motor imagery BCIs

Year: 2026

Type: article

Abstract: Objective . Machine learning classifiers used in Brain-Computer Interfaces (BCIs) make classifications based on the distribution of examples on which they were trained. When exposed to electroencephalography (EEG) from unfamiliar classes they can only make blind guesses. Instead of allowing such guesses, these out-of-distribution (OOD) samples should be detected and rejected to improve the robustness of BCIs against unfamiliar cognitive states. Approach . We study OOD detection in Motor Imagery BCIs by training a model on some classes and observing whether an unfamiliar movement class can be detected based on increased uncertainty. We tested seven different OOD detection methods and one more method that has been claimed to boost the quality of OOD detection. Main results . For many BCI users, the uncertainty for the familiar in-distribution classes can still be higher than for the OOD classes, due to the high intrinsic variability inherent in EEG signals. As a result, many OOD detection methods that have shown good performance in other machine learning domains prove to be ineffective at identifying unfamiliar motor imagery patterns in BCIs. However, we also found that OOD detection performance is correlated with on-task performance, and that Deep Ensemble models and Monte Carlo-Dropout models were able to achieve on-task Area Under the Receiver-Operator Characteristic curve > 0.9, and OOD detection ability up to 0.7. This shows that for models and subjects where task performance is high, rejecting unfamiliar cognitive states becomes feasible. Significance . Our research demonstrates a Leave-One-Class-Out OOD detection setup as an experimental paradigm for studying whether models are robust against unfamiliar motor imagery actions that were not seen in the training data. This is the first benchmark in Motor Imagery BCI that evaluates this type of OOD data. Our results demonstrate how to improve the overall safety and reliability of BCIs by preventing erroneous actions.

Source: Journal of Neural Engineering

DOI: 10.1088/1741-2552/aeac87

Authors: Merlijn Quincent Mulder, Matias Valdenegro-Toro, Andreea Ioana Sburlea, Ivo Pascal de Jong

Institution: University of Groningen

Language: English

Topic: EEG and Brain-Computer Interfaces

Subfield: Cognitive Neuroscience

Field: Neuroscience

Domain: Life Sciences

SDG: Good health and well-being

Open Access status: hybrid

Estimated APC paid: $3,325