A Comparison of Scalable Routine Clinical Materials and Observer Ratings to Assess CBT Fidelity
Year: 2024
Type: article
Abstract: Decades of research have demonstrated the efficacy of cognitive behavioral therapies (CBTs) for a wide variety of psychiatric diagnoses, resulting in the inclusion of CBT as a first-line evidence-based practice (EBP) in treatment guidelines for mood and anxiety disorders. However, some research suggests that many providers do not implement EBPs as intended. Ongoing quality monitoring is needed to support EBP implementation and sustainability, but "gold standard" fidelity monitoring (e.g. observer ratings) is time-consuming, requires extensive training, and may feel intrusive to providers and clients. In the current study, we aimed to develop a scalable method of assessing CBT fidelity that leverages information generated in routine clinical care (e.g. session worksheets and clinician checklists). Ratings of adherence based on worksheets were not correlated with ratings of adherence based on observer ratings. However, ratings of competence based on worksheets were significantly correlated with observer ratings of competence. Ratings of adherence based on clinician checklist ratings were also significantly correlated with observer-rated adherence. Results did not indicate a strong relationship between adherence or competence measured by worksheet ratings or observer and symptom change. However, adherence as measured by clinician checklists were associated with subsequent depression symptom change. Findings have a strong potential to impact fidelity monitoring strategies for a variety of CBTs. Given the limited resources and time to do full audio review in routine care settings, findings suggest that using routine materials generated in session to assess therapist competence may be a feasible alternative to the "gold standard" audio review. The trial is registered at ClinicalTrials.gov, number NCT03479398.
Source: Behaviour Research and Therapy
Authors: Amber Calloway, Torrey A. Creed, Niki B. Gumport, Cassidy Gutner, Luana Marques, Samantha Hernandez, Jiyoung Song, Clara Johnson, Soo Jeong Youn, Sohayla Elhusseini, Regine M. Deguzman-Lucero, Taylor Laskot, Heidi La Bash, Yesenia Aguilar Silvan, Caroline Cassotte, Alayna L. Park, Kimberlye Dean, Anna D. Bartuska, Booil Jo, Paul Barnett, Eric Kuhn, Robert DeRubeis, Dawne Vogt, Shannon Wiltsey Stirman
Institutions: University of Pennsylvania, Stanford University, Boston University, Massachusetts General Hospital, Rutgers, The State University of New Jersey, University of California, Berkeley, University of Washington, Harvard University, Reliant Medical Group, VA Palo Alto Health Care System, National Center for Post Traumatic Stress Disorder, University of California, Los Angeles, VA Boston Healthcare System, University of Oregon, Palo Alto Veterans Institute for Research
Language: English
Subfield: Clinical Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: hybrid
Estimated APC paid: $3,480
Awards: G3945178252, G5056611198
OpenAlex ID: W4404734857