LaCLIP-FER: Landmark-Aware Adapting Vision-Language Models for Facial Expression Recognition
Year: 2025
Type: conference-paper
Abstract: Facial emotion recognition (FER) is a crucial task for intelligent human-computer interaction, with the primary challenge being the development of discriminative facial emotion representations. However, the performance of FER is often hindered by inter-class similarity and intra-class variability, as individuals across different races, genders, and ages express emotions in diverse ways. To address these challenges, we introduce LaCLIP-FER, a method that leverages the powerful transfer capabilities of vision-language models (i.e., CLIP) while incorporating landmark information through parameter-efficient fine-tuning (PEFT). Specifically, we adapt CLIP for the FER task by designing a landmark-aware adapter (La-Adapter), which integrates invariant facial landmarks (e.g., eyes and mouth) to enhance the extraction of fine-grained facial representations without adding significant training parameters. Additionally, we introduce positive and negative prompt signals to improve the model's ability to distinguish emotional semantics and mitigate inter-class ambiguity. Experimental results on the RAF-DB and Affect-7/8 datasets demonstrate that LaCLIP-FER outperforms state-of-the-art methods, achieving WAR scores of 91.88%, 67.29%, and 63.12%, respectively.
Authors: Chuanbo Zhu, Jia Zheng, Jincai Chen
Language: English
Topic: Emotion and Mood Recognition
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: hybrid
Award: G1608827994
OpenAlex ID: W4413812499