Heterogeneous Memristive Neuromorphic Computing with Physics-Aware Cross-Modal Attention for Energy-Efficient Multimodal Emotion Recognition
Year: 2026
Type: article
Abstract: Multimodal Emotion Recognition (MER) is a crucial technology for achieving emotional intelligence in human–computer interaction. However, the substantial computational demands of MER models currently overwhelm the capacities of resource-constrained edge devices. To address this problem, this paper proposes a Heterogeneous Memristive Neural Network for Multimodal Emotion Recognition. The proposed network adopts a hardware–algorithm co-design approach, in which different memristive devices are assigned to different modal processing tasks according to their physical characteristics. Specifically, highly stable filamentary memristors are utilized to construct deterministic feature extraction pathways for text and video, whereas highly plastic Phase-Change Memory (PCM) crossbar arrays are designed for audio encoding. Based on the high plasticity of PCM, a memristive cross-modal attention network is further designed to achieve dynamic and adaptive multimodal feature fusion. Furthermore, a collaborative weight update method is proposed. During weight updates, the inherent writing noise and conductance drift of PCM are utilized as hardware-level regularization and automatic sparse learning strategies. Simulation experiments conducted on the IEMOCAP and MELD datasets using the MemTorch framework demonstrate that the proposed neural network maintains recognition accuracy comparable to GPU baseline models, even when accounting for device non-idealities. Overall, the proposed neural network effectively resolves the “performance–power” conflict when deploying high-performance MER models on resource-constrained edge devices, providing an efficient and stable neuromorphic computing solution for edge applications.
Source: Electronics
Authors: Cong Xu, Le-Jian Xu, Wei Feng, Fei Yu, Jie Jin
Language: English
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,896
OpenAlex ID: W7225891655
