Work

MRF-CMFT: Mamba-Based Quality-Gated Multi-View Fusion Framework for EEG–ECG Affect Recognition on Low-Quality Data

Year: 2026
Type: preprint
Abstract: Physiological affect recognition is inherently a multimodal fusion problem, yet its reliability is limited by noisy measurements, heterogeneous representations , and temporally varying signal quality. We propose Multi-view Representation Fusion with Quality-aware Cross-Modal Feature Transformation (MRF-CMFT), a quality-aware framework for EEG–ECG affect recognition. Each modality is represented using three complementary views: short-time Fourier transform power maps, Morlet wavelet scalograms, and compact handcrafted feature representations. These views are encoded by convo-lutional sequence encoders with a Mamba-based backbone and integrated through adaptive quality-aware view fusion. Cross-modal attention captures interactions between EEG and ECG, while a quality-aware modality gate dynamically regulates their contributions according to estimated signal reliability. A temporal Mamba aggregator then integrates consecutive windows to produce trial-level predictions. The MRF-CMFT was evaluated on the DREAMER and AMIGOS datasets using trial-grouped subject-dependent, subject-independent, and leave-one-subject-out protocols to prevent information leakage. On DREAMER, the framework achieves subject-independent AUC of 0.640, 0.534, and 0.581 for valence, arousal, and dominance, respectively ; the corresponding AMIGOS AUC are 0.496, 0.561, and 0.623. The mean subject-dependent AUC is 0.614 on DREAMER and 0.616 on AMI-GOS. Under the evaluated corruption conditions, the quality-aware fusion mechanism exhibits graceful degradation, with dominance AUC on AMIGOS remaining above 0.83 across all tested scenarios. These findings indicate that explicitly modelling representation reliability and cross-modal complementar-ity supports robust physiological affect recognition under subject variability and degraded sensing conditions.
Language: English
Open Access status: green
OpenAlex ID: W7225877451
Cites: 18
Cited by:
Related to: 10