Action-Unit-Constrained Graph Learning with Counterfactual Recourse for Auditable Facial Expression Recognition
Year: 2026
Type: article
Abstract: Facial expression recognition (FER) systems often explain predictions with saliency maps that neither identify specific facial behaviour nor indicate how a decision could change. We present AU-GRACE, which couples an action-unit (AU) relation graph with an explicit AU bottleneck to provide counterfactual recourse as small sets of plausible AU edits. Facial Action Coding System (FACS)/EMFACS knowledge supplies reliability-gated soft constraints, allowing departures from prior edges weakly supported by data. The emotion head accesses the image through the AU vector and a norm-penalised residual, so AU edits intervene in the decision. An amortised solver generates edits in 6.4 ms. AU-GRACE achieves 96.78%, 69.19%, and 94.92% accuracy on RAF-DB, AffectNet-7, and FERPlus, respectively, under the evaluation protocols defined by the dataset authors. Counterfactual validity is 94.8%, with 2.3 edited AUs and 86.4% plausibility. In a small expert study, agreement with six certified FACS coders reaches κ¯=0.71, close to their inter-coder agreement. The edits remain effective after rendering and re-classification by an independent model. These results show that, for the basic expression categories studied here, competitive recognition accuracy can be combined with AU-level recourse through an explicit AU bottleneck.
Source: Computers
Language: English
Topic: Emotion and Mood Recognition
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,172
OpenAlex ID: W7213784585