Work

Poster Session I- Poster of Distinction - A46 IMPROVING INTER-OBSERVER AGREEMENT IN ENDOSCOPIC FINDINGS WITH STANDARDIZED TERMINOLOGY

Year: 2026

Type: article

Abstract: Background A shared, standardized vocabulary for endoscopic diagnoses is essential for accurate reporting, clear communication and machine learning (ML). Heterogeneity in report language can lead to inconsistent labeling of findings, inter-observer disagreement, variation in patient management and suboptimal ML. Advances in digital imaging create opportunities to standardize imaging descriptors across providers and evaluate whether harmonized terminology reduces disagreement. Aims To assess whether standardizing diagnostic terminology decreases inter-observer disagreement for endoscopic findings. Methods Three gastroenterology (GI) residents independently assigned diagnoses to 150 endoscopic images (esophagogastroduodenoscopy, colonoscopy). An expert faculty GI at an academic center, blinded to resident assessments, reviewed the same images to establish reference diagnoses. Inter-observer disagreement between residents and the expert was calculated to generate a pre-standardization score. The residents then collaborated with the expert to develop a standardized terminology set, which was used to relabel all 150 images, and inter-observer disagreement was recalculated to generate post-standardization score. Fleiss Kappa score was then calculated to determine disagreement between four endoscopists before and after standardization. Results Overall, disagreement scores were lower for landmark recognition in esophagogastroduodenoscopy (EGD) than in colonoscopy, whereas disagreement in lesion diagnoses was lower in colonoscopy than in EGD. The largest increases in Fleiss’ kappa were observed after standardizing landmark-recognition terminology for EGDs. Details of disagreement and kappa metrics are provided in Table 1. Conclusions Standardizing terminology in endoscopy reports improves inter-observer agreement and reduces interpretive discrepancies. Agreement on major anatomic landmarks exceeded agreement on diagnostic labels for EGDs in particular, highlighting substantial variability in descriptors for endoscopic findings. Fleiss’ kappa increased most for landmark recognition in EGD, indicating that harmonized nomenclature can markedly enhance documentation consistency in this domain. Larger, multi-center studies with a broader spectrum of lesions are warranted to validate these results and to quantify the impact of report vernacular on clinical decision-making and patient outcomes. Funding Agencies None

Source: Journal of the Canadian Association of Gastroenterology

DOI: 10.1093/jcag/gwaf042.046

Authors: Y Song, S Samnani, V J Arroyo, R Gaikar, A Azad, D Armstrong

Institution: McMaster University

Language: English

Topic: Reliability and Agreement in Measurement

Subfield: Statistics, Probability and Uncertainty

Field: Decision Sciences

Domain: Social Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,198