DeepGut: A collaborative multimodal large language model framework for digestive disease assisted diagnosis and treatment
Year: 2025
Type: article
Abstract: BACKGROUND: Gastrointestinal diseases have complex etiologies and clinical presentations. An accurate diagnosis requires physicians to integrate diverse information, including medical history, laboratory test results, and imaging findings. Existing artificial intelligence-assisted diagnostic tools are limited to single-modality information, resulting in recommendations that are often incomplete and may be associated with clinical or legal risks. AIM: To develop and evaluate a collaborative multimodal large language model (LLM) framework for clinical decision-making in digestive diseases. METHODS: In this observational study, DeepGut, a multimodal LLM collaborative diagnostic framework, was developed to integrate four distinct large models into a four-tiered structure. The framework sequentially accomplishes multimodal information extraction, logical "chain" construction, diagnostic and treatment suggestion generation, and risk analysis. The model was evaluated using objective metrics, which assess the reliability and comprehensiveness of model-generated results, and subjective expert opinions, which examine the effectiveness of the framework in assisting physicians. RESULTS: The diagnostic and treatment recommendations generated by the DeepGut framework achieved exceptional performance, with a diagnostic accuracy of 97.8%, diagnostic completeness of 93.9%, treatment plan accuracy of 95.2%, and treatment plan completeness of 98.0%, significantly surpassing the capabilities of single-modal LLM-based diagnostic tools. Experts evaluating the framework commended the completeness, relevance, and logical coherence of its outputs. However, the collaborative multimodal LLM approach resulted in increased input and output token counts, leading to higher computational costs and extended diagnostic times. CONCLUSION: The framework achieves successful integration of multimodal diagnostic data, demonstrating enhanced performance enabled by multimodal LLM collaboration, which opens new horizons for the clinical application of artificial intelligence-assisted technology.
Authors: Xiao-Han Wan, Mei-Xia Liu, Yan Zhang, Guan-Jun Kou, Lei-Qi Xu, Han Liu, Xiao-Yun Yang, Xiu-Li Zuo, Yan-Qing Li
Institutions: Shandong University, Qilu Hospital of Shandong University
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: hybrid
Funders: Shandong University, Qilu Hospital of Shandong University, Sun Yat-sen University, Huazhong University of Science and Technology, Science and Technology Commission of Shanghai Municipality, Tongji University, Chinese Center for Disease Control and Prevention, Nanchang University, Zhejiang University, Shanghai Jiao Tong University, Fourth Military Medical University, Peking University, Peking Union Medical College Hospital, Peking University Third Hospital, Tianjin Medical University, Peking Union Medical College, National Health Commission of the People's Republic of China, Ruijin Hospital, Nanjing Drum Tower Hospital, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Tongji Medical College, Huazhong University of Science and Technology
Award: G1235007376
OpenAlex ID: W4413314331