Work

Cross-Modal Adaptive Feedback Knowledge Distillation for Enhanced sEMG-Based Semantic Recognition

Year: 2026

Type: article

Abstract: In semantic recognition based on surface electromyography (sEMG), low signal-to-noise ratios and physiological differences across subjects limit the ability of unimodal sEMG features to fully represent speech semantics, constraining recognition accuracy. Existing multimodal fusion methods can improve accuracy but often require large models and high computational complexity. Mismatches in feature spaces and learning capacity between multimodal teachers and unimodal students further limit cross-modal knowledge transfer. To address these issues, this paper proposed cross-modal adaptive feedback knowledge distillation (CAFKD). The multimodal teacher model uses Bi-GRU to fuse temporal features from synchronously acquired audio, lip image sequences, and sEMG signals, generating high-level representations that encode complementary semantic information. This multimodal knowledge was transferred to an sEMG-only student through a joint loss combining classification, output distribution distillation, and intermediate feature distillation. In addition, the cross-modal adaptive feedback mechanism (CAF) periodically adjusted the teacher using stagewise output feedback from a temporary student, enabling teacher guidance to better match the student’s learning state. In subject-independent evaluation on the AVE-Speech dataset, CAFKD achieved 78.13% accuracy on the 101-class closed-set short-utterance classification task, exceeding the unimodal Bi-GRU baseline by 4.82 percentage points. Ablation experiments showed that adding intermediate feature distillation to output distribution distillation increased test accuracy by 1.49 percentage points. CAF further improved accuracy by 1.46 percentage points over joint distillation. The student model also retains the same parameter count and FLOPs as the Bi-GRU baseline, at 4.42 M and 0.29 G, respectively. These results indicate that CAFKD improves recognition performance on unseen subjects while preserving a lightweight architecture for sEMG-only inference.

Source: Big Data and Cognitive Computing

DOI: 10.3390/bdcc10110368

Authors: Junfeng Wei, Guihong Ruan, Peiguang Jing, Yun Li

Institutions: Guangxi Minzu University, Tianjin University, Guangxi University

Language: English

Topic: Muscle activation and electromyography studies

Subfield: Biomedical Engineering

Field: Engineering

Domain: Physical Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,172