Inter-Model Agreement of ChatGPT, Claude, and Grok for IBO-Based Lateral Cephalometric Measurements: An Exploratory Comparative Study
Year: 2026
Type: article
Abstract: Background: General-purpose multimodal artificial intelligence (AI) systems can analyze radiographic images, but their numerical consistency for quantitative orthodontic measurements is unknown. Objective: To compare the inter-model agreement of ChatGPT, Claude, and Grok when estimating standardized Italian Board of Orthodontics (IBO) cephalometric measurements on the same lateral cephalograms. Methods: Twenty pre-existing lateral cephalograms from 20 patients aged 10-60 years were analyzed by ChatGPT, Claude, and Grok with the same standardized prompt. Twelve IBO parameters were requested. Three linear measurements were excluded because metric calibration was not validated, leaving nine angular parameters. Model-specific descriptive statistics, pairwise mean absolute differences (MAD), root mean square error, and two-way absolute-agreement single-measure intraclass correlation coefficients (ICC) were calculated. Friedman tests with Holm correction assessed systematic differences between models. Results: Complete-case sample size ranged from 16 to 20 across angular parameters. ICCs were poor for all nine measurements (0.03-0.26). Pooled pairwise MADs were 2.14 degrees for ChatGPT-Claude, 2.71 degrees for ChatGPT-Grok, and 2.59 degrees for Claude-Grok. No angular parameter showed a statistically significant systematic difference among models after Holm correction. Conclusions: ChatGPT, Claude, and Grok produced broadly similar group-level cephalometric estimates but poor case-level numerical agreement. General-purpose multimodal AI systems should not be considered interchangeable cephalometric measurement tools. Validation against an expert human reference standard is required before clinical use.
Language: English
Subfield: Orthodontics
Field: Dentistry
Domain: Health Sciences
Open Access status: diamond
OpenAlex ID: W7225814577