Evaluation of large language models in pediatric dentistry: a Bloom’s taxonomy-based analysis
Year: 2025
Type: article
Abstract: Aim : This study aimed to evaluate the performance of three large language models (LLMs)—ChatGPT-4.0, Claude 3.5 Sonnet, and DeepSeek R1—in answering multiple-choice questions (MCQs) related to pediatric dentistry. Accuracy and justification quality were analyzed using Bloom’s taxonomy. Materials and methods : A total of 90 MCQs were developed based on the American Academy of Pediatric Dentistry (AAPD) guidelines, ensuring cognitive diversity across Bloom’s taxonomy levels. The models were assessed for answer accuracy and required to provide justifications, which were scored using a structured 4-point rubric by two independent pediatric dentistry experts. Statistical analyses, including Kruskal-Wallis tests and one-way ANOVA, were used to compare performance. Results : DeepSeek R1 demonstrated the highest accuracy (92.2%), followed by Claude 3.5 sonnet (86.6%) and ChatGPT-4.0 (72.2%). Significant differences in accuracy were observed at the “Understanding” level ( p =0.009). Justification quality also varied significantly among models, with DeepSeek R1 outperforming the others ( p <0.001). Inter-rater reliability was high (ICC: 0.615–0.848), showing the reliability of the evaluation. Conclusion : The study shows variations in LLM performance, with DeepSeek R1 excelling overall. It holds promise for pediatric dentistry education and AI decision-making, but further improvements are needed for better reasoning and clinical use.
Source: Folia Medica
Authors: Atanu Mukhopadhyay, Santanu Mukhopadhyay, Raju Biswas
Institutions: Amazon (United States), Malda Medical College and Hospital
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: diamond
OpenAlex ID: W4413971561