Facial Expression Recognition Through Joint Modeling of Local Visual Representations and Continuous Affect
Year: 2026
Type: article
Abstract: Facial expressions are an important means of conveying human emotions. Accurately distinguishing similar expressions requires capturing subtle appearance differences across facial regions, while a single class label cannot fully represent the continuous affective relationships among expression categories. To this end, we propose a Valence-Arousal-Guided Region Attention Network (VA-RANet) for joint discrete expression classification and continuous valence-arousal (VA) estimation. The model learns multiple spatial weight maps from shared features, constructs regional representations through weighted aggregation, and fuses them with global features. A region complementarity constraint is employed to limit the similarity among the spatial weight maps and encourage differentiated regional responses. During training, the distances between the samples and the fixed class prototypes in the VA space are converted into a probability distribution and subsequently fused with the discrete expression labels to generate soft labels that provide auxiliary supervision for discrete expression classification. Ground-truth VA annotations are used for AffectNet, whereas VA pseudo-labels obtained through transfer are used for RAF-DB and CAER-S. Only facial images are required at inference. The model achieves classification accuracies of 65.71% and 62.93% under the seven-class and eight-class AffectNet settings, respectively. Experimental results indicate that the combination of differentiated multi-region visual representations and VA-guided supervision contributes to improved facial expression recognition performance.
Source: Sensors
DOI: 10.3390/s26227279
Authors: Pujun Xue(Conceptualization, Data curation, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing), Qigao Sun(Conceptualization, Investigation, Validation, Writing – review & editing), Wei Xu(Investigation, Validation, Writing – review & editing), Mufang Lin(Data curation, Visualization, Writing – review & editing), Angelo Cangelosi(Supervision, Writing – review & editing), Huili Cai(Writing – review & editing)
Language: English
Topic: Emotion and Mood Recognition
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W7236465799
