Can LLM Agents Really Simulate Clients in Counseling?
Year: 2026
Type: conference-paper
Abstract: Recent studies have increasingly examined the use of large language models (LLMs) to simulate clients in counseling sessions, yet many report that LLM-generated responses tend to be excessively positive, reducing the realism of therapeutic interactions. This study conducts a quantitative analysis to verify whether such positivity bias consistently appears across a range of closed-source and open-source models and to evaluate strategies intended to mitigate it. To accomplish this, we developed a dataset and an evaluation model that score client utterances along sentiment and stance dimensions grounded in clinical expert criteria. Using multiple LLMs combined with various bias mitigation methods, we generated artificial counseling dialogues and compared them with real client data. Experimental findings show that most LLM clients exhibit significant positivity bias, revealing clear statistical differences from real counseling conversations. Although the proposed mitigation strategies alleviated this bias to some degree, they did not fully close the gap. The findings demonstrate the presence of a quantifiable gap between LLM-based counseling simulations and real client responses, suggesting the development of standardized evaluation frameworks, session-context-adaptive bias mitigation, and persona design methods to bridge this gap. All code and generated data used in this study are publicly available on our GitHub repository. 1
Institution: Sungkyunkwan University
Language: English
Topic: Speech and dialogue systems
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7163994787