Year: 2025
Type: article
Abstract: BACKGROUND: Histopathological interpretation is crucial for diagnosing inflammatory bowel disease (IBD), distinguishing between Crohn's Disease (CD), Ulcerative Colitis (UC), IBD-Unclassified (IBD-U), and Non-IBD colitis (NIBDC). However, interobserver variability and limited expertise can reduce diagnostic accuracy. Large Language Models (LLMs) such as GPT-5 may offer clinical support in interpreting histology reports. METHODS: We analyzed 100 real-life histological reports from ileo-colonoscopies, equally representing CD, UC, IBD-U, and NIBDC, collected across five Italian healthcare centers, including both IBD-specialized and non-specialized hospitals. A reference standard was established by an expert pathologist. Independent classifications were generated by GPT-5, five gastrointestinal pathologists, five IBD-expert gastroenterologists (GIs), and five non-expert GIs. Diagnostic performance (accuracy, recall, precision, F1-score), agreement with the reference standard (Cohen's κ), and inter-rater reliability (Fleiss' κ) were assessed. RESULTS: GPT-5 achieved the highest agreement with the reference standard with the highest accuracy (76.0%), compared to pathologists (68.6%), IBD-experts (69.2%), and non-experts (63.2%). Agreement with the reference standard was substantial for GPT-5 (κ = 0.671) and moderate for human groups (κ = 0.508-0.588). GPT-5 showed perfect recall for CD and UC, high recall for NIBDC (96.0%), but poor performance for IBD-U (recall 8.0%, F1-score 14.3%). Fleiss' κ indicated moderate agreement among pathologists and IBD-experts, and fair agreement among non-experts. CONCLUSION: GPT-5 demonstrated reliable performance in interpreting IBD histological reports, exhibiting high accuracy and strong agreement with the reference standard. While unreliable for IBD-U, GPT-5 may serve as a supportive tool in histopathological interpretation of IBD, particularly in centers with limited access to expert pathologists or IBD-specialists.
DOI: 10.1002/ueg2.70161
Authors: Marcello Maida, Alessandro Vitello, Fabio Salvatore Macaluso, Marco Daperno, Giammarco Mocci, Antonio Rispo, Giulio Calabrese, Nicola L. Decarli, Lucrezia Laschi, Caterina Fattorini, Giorgia Locci, Rachele Del Sordo, Dario Ligresti, Matteo Tacelli, Manuele Furnari, Sandro Sferrazza, Giovanni Marasco, Antonio Facciorusso, Ambrogio Orlando, Vincenzo Villanacci
Institutions: Università degli Studi di Enna Kore, Ospedale Vincenzo Cervello, A. O. Ordine Mauriziano di Torino, Azienda Ospedaliera G. Brotzu, Federico II University Hospital, University of Naples Federico II, Ospedale Misericordia - Grosseto, Nuovo Ospedale San Giovanni di Dio, Azienda USL Toscana Nord Ovest, University of Perugia, Istituto Mediterraneo per i Trapianti e Terapie ad Alta Specializzazione, IRCCS Ospedale San Raffaele, Istituto di Ricovero e Cura a Carattere Scientifico San Raffaele, Ospedale Policlinico San Martino, University of Genoa, Azienda di Rilievo Nazionale ed Alta Specializzazione, IRCCS Azienda Ospedliero-Universitaria di Bologna Policlinico di Sant'Orsola, University of Bologna, University of Salento, Azienda Socio Sanitaria Territoriale degli Spedali Civili di Brescia, University of Brescia
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: hybrid
Estimated APC paid: $2,930
OpenAlex ID: W4417429691