Work

Large language models for orthodontic classification using intraoral digital models: a diagnostic accuracy and reproducibility study

Year: 2026
Type: article
Abstract: Background Multimodal artificial intelligence (AI) systems built on large language models are consulted by patients and clinicians, but their performance on visual orthodontic classification is not well characterized. This study evaluated three such systems for agreement with an expert orthodontist reference standard, category-specific diagnostic performance and four-week reproducibility, using intraoral digital scan exports. Methods In this retrospective diagnostic accuracy study, 90 patients were selected from archived records with equal numbers in each Angle class ( n = 30). Five standardized two-dimensional views exported from each iTero scan were submitted to Claude Opus 4.7, GPT-5.5 and Gemini 3 Flash with one zero-shot prompt. Two orthodontists independently classified each case from the complete scan record with metric measurements; their agreed categories formed the reference standard. Sagittal classification, vertical relationship, crowding severity and arch form were assessed. Twenty patients were reassessed after four weeks, and the orthodontists also classified all cases from the same five images. Accuracy, class-specific sensitivity and specificity, macro-average recall and Cohen’s kappa were calculated with 95% confidence intervals (CIs). Results Overall accuracy ranged from 29.2% (Gemini 3 Flash, arch form) to 64.0% (GPT-5.5, arch form), macro-average recall from 38.7% (Gemini 3 Flash, arch form) to 60.9% (GPT-5.5, vertical relationship), and unweighted kappa from 0.044 (Gemini 3 Flash, arch form) to 0.299 (Gemini 3 Flash, crowding severity). Class III sensitivity was 10.0–13.3%, compared with 60.0–80.0% for Class I. Classifying the same five images, the orthodontists agreed with the reference standard in 96.6–100% of cases. Test–retest kappa ranged from − 0.238 (Gemini 3 Flash, sagittal classification) to 0.433 (GPT-5.5, vertical relationship), with most CIs including zero; in the retest subsample, sagittal accuracy fell from 55.0 to 60.0% to 5.0–35.0%. Each orthodontist reproduced at least 19 of 20 classifications for each parameter. Conclusions Under the conditions tested, the three systems showed low agreement with the expert reference standard, very low sensitivity for Class III malocclusion and limited four-week reproducibility. Overall accuracy did not reflect category-specific performance, and reproducibility needs to be evaluated separately from accuracy. The systems tested are not suitable for standalone orthodontic diagnosis under these conditions.
Language: English
Subfield: Orthodontics
Field: Dentistry
Open Access status: gold
Estimated APC paid: $3,190
OpenAlex ID: W7232204820
FWCI:
Cites:
Cited by:
Related to: 10