When AI speaks like a specialist: ChatGPT-4 in the management of inflammatory bowel disease
Year: 2025
Type: article
Abstract: Background: Artificial intelligence (AI) is gaining traction in healthcare, especially for patients' education. Inflammatory bowel diseases (IBD) require continuous engagement, yet the quality of online information accessed by patients is inconsistent. ChatGPT, a generative AI model, has shown promise in medical scenarios, but its role in IBD communication needs further evaluation. The objective of this study was to assess the quality of ChatGPT-4's responses to common patient questions about IBD, compared to those provided by experienced IBD specialists. Methods: Twenty-five frequently asked questions were collected during routine IBD outpatient visits and categorized into five themes: pregnancy/breastfeeding, diet, vaccinations, lifestyle, and medical therapy/surgery. Each question was answered by ChatGPT-4 and by two expert gastroenterologists. Responses were anonymized and evaluated by 12 physicians (six IBD experts and six non-experts) using a 5-point Likert scale across four dimensions: accuracy, reliability, comprehensibility, and actionability. Evaluators also attempted to identify whether responses were AI- or human-generated. Results: < 0.001). The best-rated scenarios were medical therapy and surgery; the diet scenario consistently received lower scores. Only 33% of AI-generated responses were correctly identified as such, indicating strong similarity to human-written answers. Both expert and non-expert evaluators rated AI responses highly, though IBD specialists gave higher ratings overall. Conclusion: ChatGPT-4 generated high-quality, clear, and actionable responses to IBD-related patient questions, often outperforming human experts. Its outputs were frequently indistinguishable from those written by physicians, suggesting potential as a supportive tool for patient education. Nonetheless, further studies are needed to assess real-world application and ensure appropriate use in personalized clinical care.
Authors: Elena De Cristofaro, Francesca Zorzi, Maria Abreu, Alice Colella, Giovanna Del Vecchio Blanco, Gionata Fiorino, Elisabetta Lolli, Nurulamin Noor, Loris Riccardo Lopetuso, Mathieu Pioche, Jean Grimaldi, Omero Alessandro Paoluzi, Joana Roseira, Giorgia Sena, Edoardo Troncone, Emma Calabrese, Giovanni Monteleone, Irene Marafini
Institutions: Policlinico Tor Vergata, University of Miami, University of Rome Tor Vergata, Azienda Ospedaliera San Camillo-Forlanini, University of Cambridge, Cambridge University Hospitals NHS Foundation Trust, Agostino Gemelli University Polyclinic, Istituti di Ricovero e Cura a Carattere Scientifico, Hospices Civils de Lyon, Hôpital Edouard Herriot, Algarve Biomedical Center, Unidade Local de Saúde do Algarve
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $2,353
OpenAlex ID: W4415017401