Reliability in Judging Basic Emotions: Lessons from Session Transcripts
Year: 2026
Type: article
Abstract: Background/Objectives: Psychotherapists have become increasingly interested in identifying how recurrent emotional themes might add value to the clinical process, especially in the context of recent developments in affective neuroscience. However, much of this literature remains anecdotal, and it is unclear whether therapists can reliably identify these emotions in routine practice, especially in the context of a free-flowing narrative. In a parallel development, large language models (LLMs) have recently been deployed to classify emotion from narrative texts, though this literature has yet to be aligned with the findings of modern affective neuroscience. Methods: The present study addresses this gap directly and, to our knowledge, represents the first attempt to code psychotherapy transcripts using both independent human raters and an AI coding methodology. The data were drawn from two remarkable patients with focal brain lesions, both with profound amnesia, in long-term psychotherapy. In an exploratory investigation, one session from each patient was coded independently by four clinically experienced psychotherapists, and separately by a single large language model operating under two different prompting conditions. Transcripts were coded for the presence of basic emotional systems. Results: The data suggest that overall inter-rater agreement about basic emotions was poor for both human and AI coders. Conclusions: The present study offers proof-of-concept that this approach is of scientific as well as clinical interest. The findings suggest that greater rigour needs to be applied to the way in which coders are prepared for, and implement, the classification of basic emotion systems.
Source: Psychiatry International
Language: English
Subfield: Clinical Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $1,207
OpenAlex ID: W7221513129