Work

The reliability of answers about root canal retreatment provided by artificial intelligence chatbots

Year: 2026

Type: article

Abstract: Background Easily accessible artificial intelligence (AI)-based chatbots can serve as a consulting tool for patients before meeting with a dentist. This study aimed to evaluate and compare the reliability of answers provided by different AI chatbots to frequently asked questions in the field of root canal retreatment. Methods The questions were obtained from the most frequently asked questions about retreatment by patients to endodontists (n = 10) and ChatGPT-4 (OpenAI) (n = 10). The 20 questions were asked once to 5 different AI chatbots (ChatGPT-4, ChatGPT-4o [OpenAI], Gemini 1.5 Flash [Google], Gemini 1.5 Pro [Google], DeepSeek-V.3 [High-Flyer]), and the responses (n = 100) were independently evaluated by 3 researchers using a modified Global Quality Score (GQS), modified DISCERN (mDISCERN), and The Completeness, Lack of Misinformation, Evidence, Appropriateness, Relevance (CLEAR) tools. Interrater reliability was assessed via the intraclass correlation coefficient. Data were analyzed using Kruskal-Wallis H and post hoc Mann-Whitney U tests because of nonnormal distribution ( P < .05). Results High interrater agreement was observed (intraclass correlation coefficient, 0.87-0.94). Significant differences were identified across all platforms. ChatGPT-4o and Gemini 1.5 Pro consistently outperformed all other models across GQS, mDISCERN, and CLEAR. Conversely, Gemini 1.5 Flash showed the significantly lowest performance ( P < .05). DeepSeek-V3 achieved significantly higher mDISCERN scores than ChatGPT-4 despite lower GQS scores than the top-performing models. Strong positive correlations were found among all evaluation scales ( r = 0.67-0.73; P < .05). Conclusions ChatGPT-4o and Gemini 1.5 Pro showed the highest performance on GQS, mDISCERN, and CLEAR, serving as reliable tools in terms of accuracy rates and usable as trustworthy tools in patient education regarding retreatment. AI-based chatbots can answer patient questions and provide health-related information, but they cannot replace professional medical advice.

Source: JADA Foundational Science

DOI: 10.1016/j.jfscie.2026.100081

Authors: Gamze Ebrar Nalbant, Tansu Merve Besparmak, Şeyma Atli, Ali Turkyilmaz, Ali Erdemir

Institution: Kırıkkale University

Language: English

Topic: Artificial Intelligence in Healthcare and Education

Subfield: Health Informatics

Field: Medicine

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,500