Intergrader agreement for diabetic retinopathy (DR) using hand-held retinal imaging
Year: 2021
Type: conference-paper
Study design: Observational Study
Abstract: Purpose : To assess intergrader agreement when interpreting retinal images acquired with handheld retinal imaging devices for DR screening. Methods : Mydriatic retinal images acquired using 3 hand-held retinal cameras [Aurora (AU), Smartscope (SS), RV700 (RV)] were compared with Early Treatment Diabetic Retinopathy Study 7-field standard photography (ETDRS photos). All handheld retinal images were independently evaluated by 2 grader-certified nonphysician staff and 1 retina specialist (RPS). The ETDRS photos were evaluated by a senior retina specialist (PSS). All graders had completed a structured certification program in DR assessment. Grading was performed using the International Classification for DR. Agreement was measured using kappa (k) statistics, multirater k across graders and sensitivity/specificity was calculated for vision threatening DR [(vtDR): severe NPDR or worse]. Results : Images from 177 eyes of 92 patients with diabetes were evaluated. Severity by ETDRS photos: no DR 40.1%, mild NPDR 19.2%, moderate 14.7%, severe 10.2%, proliferative DR 15.8%. Ungradable rate for DR was AU: 0%; SS: 4.5%; RV: 4.0%; and ETDRS: 0%. Results are presented in table 1. Multirater κ for DR severity (0.58 – 0.65) and vtDR (0.71 – 0.74) was uniform across all devices. Agreement (κ) for DR severity between ETDRS photos and graders was similar across nonphysician graders across devices (0.46 – 0.48) and highest with retina specialist evaluation across devices (0.70-0.78). Sensitivity/specificity for vtDR on ETDRS photo for nonphsician graders was over 0.95 in all devices but specificity was 0.87-0.89. Retina specialist specificity for vtDR was 0.96-0.97. Conclusions : In assessing DR severity, substantial over-all agreement (0.58-0.74) among all graders was achieved emphasizing the benefit of a structured program of grading certification. Retinal images obtained using all 3 handheld cameras achieved the highest agreement with ETDRS photos (0.70-0.78) when evaluated by a retinal specialist. Among the nonphysician graders for all devices, sensitivity for vtDR was over 90% but specificity remained 87-89%. These findings suggest that in DR screening programs using hand-held retinal imaging, secondary grading of eyes with vtDR by more experienced graders may be necessary to minimize unnecessary referrals that would otherwise decrease the effectiveness of screening programs.
Institutions: Joslin Diabetes Center, Harvard University
Language: English
Topic: Retinal Imaging and Analysis
Subfield: Radiology, Nuclear Medicine and Imaging
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W3182336232