Work

Performance of large language models in early orthodontic treatment: a comparative study of ChatGPT-5.4 and deepSeek-V3.2

Year: 2026

Type: article

Abstract: Large language models (LLMs) are increasingly being used by patients and dental professionals to obtain orthodontic information. However, few studies have specifically compared the performance of LLMs in answering questions related to early orthodontic treatment, an area that requires complex clinical reasoning and high-quality patient education. This study aimed to evaluate the clinical applicability of ChatGPT-5.4 and DeepSeek-V3.2 in early orthodontic treatment using a multidimensional evaluation framework. Five orthodontists developed 20 questions across five domains (age of treatment, adverse oral habits, early treatment, complications, and retention and relapse). The standard answers were derived from clinical guidelines, authoritative textbooks, and articles published in Q1 journal. Each question was submitted once to each LLMs. Responses were anonymised and independently evaluated by five blinded orthodontists. Accuracy and comprehensiveness were assessed using a five-point Likert-type framework combined with visual analogue scale (VAS)-based continuous scoring. Readability was calculated using the SMOG formula. References were cross-checked against five databases and were considered authentic when their bibliographic details corresponded to an indexed publication in at least one database. Thinking time, generation time, and word count were recorded. Inter-rater agreement was assessed using the intraclass correlation coefficient (ICC). Depending on the distribution of paired differences, paired t-tests or Wilcoxon signed-rank tests were applied. ChatGPT-5.4 showed significantly higher accuracy (9.03 ± 0.34 vs. 7.87 ± 0.95, p < 0.001) and comprehensiveness (4.49 ± 0.19 vs. 3.87 ± 0.54, p < 0.001) than DeepSeek-V3.2. DeepSeek-V3.2 was significantly faster (thinking time: 7.0 vs. 82.9 s; generation time: 39.4 vs. 133.4 s, both p < 0.001) and more readable (SMOG grade: 13.48 vs. 14.78, p = 0.003). No significant differences were observed for word count ( p = 0.306) and reference authenticity ( p = 0.895). In the domain analysis, ChatGPT-5.4 significantly outperformed DeepSeek-V3.2 in the domains of early treatment and retention and relapse ( p < 0.05), whereas no significant differences were observed in the age of orthodontic treatment, adverse oral habits, and complications domains. Inter-rater agreement was good (ICC = 0.767–0.801). ChatGPT-5.4 demonstrated greater accuracy and comprehensiveness, whereas DeepSeek-V3.2 generated responses more rapidly and produced more readable text. However, neither model achieved a readability level appropriate for the average patient. Neither model should replace professional clinical judgment.

Source: BMC Oral Health

DOI: 10.1186/s12903-026-09662-z