Work

Comparison of large language models in answering early gastric cancer clinical problems and recognizing multimodal endoscopic images

Year: 2026

Type: article

Abstract: Background: Large language models (LLMs) show promise in medical information support, but their efficacy in early gastric cancer (EGC) remains under-evaluated. This study systematically assesses LLMs performance in answering EGC clinical questions and analyzing white light endoscopy (WLE) and narrow band imaging (NBI) images. Methods: This study systematically reviewed clinical guidelines and interviewed patients to compile 40 clinical questions and 50 Single-choice questions related to EGC. Ten clinicians and ten EGC patients quantitatively scored LLM-generated responses across four dimensions: accuracy, correlation, completeness, and Comprehension. Readability metrics and Single-choice answer accuracy rates were further calculated. Additionally, the study incorporated 482 white light endoscopy (WLE) and 478 narrow band imaging (NBI) images from four hospitals in China. It systematically evaluated the diagnostic efficacy of different LLMs for multi-task recognition of EGC based on accuracy, sensitivity, specificity, and area under the curve (AUC). Findings: In text assessment, ChatGPT-4o achieved the highest scores for accuracy (86), correction (88), and comprehension (88), while Grok-3 scored highest for completeness (90). The accuracy rates for ChatGPT-4o, DeepSeek-V3.2, Gemini-2.5, and Grok-3 in answering Single-choice questions were 78%, 88%, 76%, and 82%, respectively. In image assessment, ChatGPT-5.2 demonstrated an accuracy of 62.33% (95% CI, 56.58–67.84), sensitivity of 86.77% (95% CI, 81.10–91.25), specificity of 0.57 (95% CI, 0.48–0.66), and an AUC of 20.72% (95% CI, 13.61–29.45) for identifying EGC in WLE images. 20.72% (95% CI, 13.61–29.45), 0.57 (95% CI, 0.48–0.66), respectively. For NBI images, the accuracy, sensitivity, specificity, and AUC were 58.00% (95% CI, 52.19–63.65), 88.41% (95% CI, 82.50-92.88), 21.32% (95% CI, 14.77-29.17), 0.50 (95% CI, 0.41-0.59). Gemini-3.0 pro demonstrated an accuracy of 83.67% (95% CI, 78.99–87.67),, 83.95% (95% CI, 83.84–84.05), 0, 0.50 (95% CI, 0.41–0.59), respectively. On NBI images, the accuracy, sensitivity, specificity, and AUC were 58.00% (95% CI, 52.19–63.65), 88.41% (95% CI, 82.50-92.88), 21.32% (95% CI, 14.77-29.17), 0.50 (95% CI, 0.41-0.59). Grok-4 demonstrated accuracy, sensitivity, specificity, and AUC values of 73.33% (95% CI, 67.95–78.25), 85.83% (95% CI, 80.77–89.99), 23.33% (95% CI, 13.38-36.04), 0.56 (95% CI, 0.46-0.65). On NBI images, the accuracy, sensitivity, specificity, and AUC were 78.33% (95% CI, 73.24-82.86), 83.27% (95% CI, 78.39-87.44), 5.26% (95% CI, 0.13-26.03), and 0.48 (95% CI, 0.39-0.56), respectively. Interpretation: LLMs has shown potential in addressing clinical issues related to EGC, but exhibits significant heterogeneity in text evaluation. In addition, the ability of LLMs to diagnose EGC in multimodal endoscopic images is still limited. Future efforts should focus on optimizing the structure of medical knowledge, enhancing multimodal understanding, and promoting clinical translation.

Source: BMC Gastroenterology

DOI: 10.1186/s12876-026-05474-1