Not one size fits all: model-specific effects of reasoning on emotion recognition from body-gesture videos
Year: 2026
Type: article
Abstract: Background The growing use of generative artificial intelligence (GAI) in mental-health contexts underscores the need to evaluate its emotion-recognition capacity. Objective This study examined whether a reasoning-oriented model or structured Chain-of-Thought (CoT) prompting improved recognition of emotions conveyed through bodily gestures. Methods Four Gemini models were evaluated using 36 silent videos depicting 18 emotions. Each model completed 20 trials per video under Zero-Shot and CoT prompting, yielding 5,760 responses. Population-averaged generalized estimating equation models compared accuracy across models, prompts, emotional groups, and the human benchmark. Results Gemini 2.0 Pro showed higher accuracy than the other models following multiplicity adjustment, whereas Gemini 2.0 Flash Thinking did not outperform the general-purpose models. The marginal CoT-versus-Zero-Shot contrast was not statistically significant. However, the Model × Prompt Type interaction was significant. Accuracy was higher under the tested CoT condition than under Zero-Shot prompting for Gemini 2.0 Pro and lower under the tested CoT condition for Gemini 2.0 Flash. The Model × Prompt Type × Emotional Group interaction was not significant. The Performer × Emotional Group interaction was significant, although within-model differences between positive and negative emotions did not survive adjustment. Three models showed lower accuracy than humans for negative emotions, whereas no significant model–human differences were found for positive emotions. Conclusion The data did not support a general improvement in emotion recognition under CoT prompting. Prompt-condition differences varied across models. This pattern underscores the need to evaluate reasoning prompts for specific models and tasks before application in mental-health contexts.
Source: Frontiers in Psychology
Authors: Hadar Dery(Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing), David Piterman(Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing), Gunther Meinlschmidt(Supervision, Writing – review & editing), Kfir Bar(Supervision, Writing – review & editing), Alon Geller(Data curation, Formal analysis, Software, Visualization, Writing – review & editing), dorit Hadar Shoval(Writing – review & editing), Mosh Levy(Writing – review & editing), Elad Refoua(Conceptualization, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing), Zohar Elyoseph(Conceptualization, Investigation, Methodology, Resources, Supervision, Writing – original draft, Writing – review & editing)
Language: English
Topic: Emotion and Mood Recognition
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,801
OpenAlex ID: W7125631838
