Year: 2024
Type: article
Abstract: Purpose: The aim is to assess GPT-4V's (OpenAI) diagnostic accuracy and its capability to identify glaucoma-related features compared to expert evaluations. Design: Evaluation of multimodal large language models for reviewing fundus images in glaucoma. Subjects: A total of 300 fundus images from 3 public datasets (ACRIMA, ORIGA, and RIM-One v3) that included 139 glaucomatous and 161 nonglaucomatous cases were analyzed. Methods: Preprocessing ensured each image was centered on the optic disc. GPT-4's vision-preview model (GPT-4V) assessed each image for various glaucoma-related criteria: image quality, image gradability, cup-to-disc ratio, peripapillary atrophy, disc hemorrhages, rim thinning (by quadrant and clock hour), glaucoma status, and estimated probability of glaucoma. Each image was analyzed twice by GPT-4V to evaluate consistency in its predictions. Two expert graders independently evaluated the same images using identical criteria. Comparisons between GPT-4V's assessments, expert evaluations, and dataset labels were made to determine accuracy, sensitivity, specificity, and Cohen kappa. Main Outcome Measures: The main parameters measured were the accuracy, sensitivity, specificity, and Cohen kappa of GPT-4V in detecting glaucoma compared with expert evaluations. Results: GPT-4V successfully provided glaucoma assessments for all 300 fundus images across the datasets, although approximately 35% required multiple prompt submissions. GPT-4V's overall accuracy in glaucoma detection was slightly lower (0.68, 0.70, and 0.81, respectively) than that of expert graders (0.78, 0.80, and 0.88, for expert grader 1 and 0.72, 0.78, and 0.87, for expert grader 2, respectively), across the ACRIMA, ORIGA, and RIM-ONE datasets. In Glaucoma detection, GPT-4V showed variable agreement by dataset and expert graders, with Cohen kappa values ranging from 0.08 to 0.72. In terms of feature detection, GPT-4V demonstrated high consistency (repeatability) in image gradability, with an agreement accuracy of ≥89% and substantial agreement in rim thinning and cup-to-disc ratio assessments, although kappas were generally lower than expert-to-expert agreement. Conclusions: GPT-4V shows promise as a tool in glaucoma screening and detection through fundus image analysis, demonstrating generally high agreement with expert evaluations of key diagnostic features, although agreement did vary substantially across datasets. Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Source: Ophthalmology Science
Language: English
Topic: Retinal Imaging and Analysis
Subfield: Radiology, Nuclear Medicine and Imaging
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $2,500
Awards: G1162459402, Multi-modal Health Information Technology Innovations for Precision Management of Glaucoma, G1429536016, G1513198031, G1610538956, G2070653470, G2179675123, G2893845237, G3753064694, G3783232471, G4580167800, G4863971304, African Descent and Glaucoma Evaluation (ADAGES) II: Glaucoma Progression, G5108157574, G545054580, G5498426564, G5873106719, G6255259697, G6613626801, G6781972596, G7389831629, G8009587674, G8453215056, G8557029261, G8786233291, G8817645623, G9000032791, G9000079772, G9000095606, G9000098550, G9000119887, G9000134718, G9000148692, G9000162136, G9000242127, G9000264818, Short-Term Research training In Vision and Eye health (STRIVE), Multimodal Artificial Intelligence to Predict Glaucomatous Progression and Surgical Intervention, G909112076
OpenAlex ID: W4404850869