Year: 2025
Type: conference-paper
Abstract: In this study, we address the challenge of consistently following emotional support strategies in long conversations by large language models (LLMs).We introduce the Strategy-Relevant Attention (SRA) metric, a model-agnostic measure designed to evaluate the effectiveness of LLMs in adhering to strategic prompts in emotional support contexts.By analyzing conversations within the Emotional Support Conversations dataset (ESConv) using LLaMA models, we demonstrate that SRA is significantly correlated with a model's ability to sustain the outlined strategy throughout the interactions.Our findings reveal that the application of SRAinformed prompts leads to enhanced strategic adherence, resulting in conversations that more reliably exhibit the desired emotional support strategies over longer conversations.Furthermore, we contribute a comprehensive, multibranch synthetic conversation dataset for ES-Conv, featuring a variety of strategy continuations informed by our optimized prompting method.The code and data are publicly available on our Github.1
Authors: Ethan Braverman, Vittesh Maganti, Nysa Lalye, Akhil Ganti, Michael Lu, Kevin Zhu, Vasu Sharma, Sean O’Brien
Language: English
Subfield: Social Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: gold
OpenAlex ID: W4412944111