A deep learning based automatic report generator for retinal optical coherence tomography images
Year: 2025
Type: article
Abstract: Reading and summarizing insights from Optical Coherence Tomography (OCT) images is a routine yet time-consuming task that requires expensive time from experienced ophthalmologists. This paper introduces the Multi-label OCT Report Generation (MORG) model, a deep learning approach to assist in the interpretation of OCT images. MORG employs dual image encoders to extract features from OCT image pairs, fusing them through a multi-scale module with an attention mechanism, followed by a sentence decoder to produce reports. Trained and tested on 57,308 retinal OCT image pairs, MORG achieved high classification accuracy for 16 pathologies with 37 descriptive types. It also excelled in a blind grading test against general large language models and other state-of-the-art image captioning models, scoring 4.55 compared to ophthalmologists' 4.63 out of a maximum of 5. Furthermore, MORG has the potential to reduce the report drafting time for ophthalmologists by 58.9%, significantly alleviating their workload.
Source: npj Digital Medicine
Authors: Xinjian Chen, Huazhu Fu, Jingtao Wang, Tian Lin, Qian Cheng, Cangxin Li, Meng Wang, Zhongyue Chen, Aidi Lin, Anlin Zhang, Weifang Zhu, Shirong Chen, Fei Shi, Dehui Xiang, Baoqing Nie, Yi Zhou, Yuanyuan Peng, Danqi Fang, Chao Guo, Ting Wang, Mingzhi Zhang, Chi Pui Pang, Haoyu Chen
Institutions: University of Electronic Science and Technology of China, Soochow University, Sichuan Academy of Medical Sciences & Sichuan Provincial People's Hospital, Agency for Science, Technology and Research, Institute of High Performance Computing, Chinese University of Hong Kong, Shantou University, Joint Shantou International Eye Center, Chinese University of Hong Kong, Shenzhen
Language: English
Topic: Retinal Imaging and Analysis
Subfield: Radiology, Nuclear Medicine and Imaging
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $4,090
OpenAlex ID: W4415359053