2025 Expert consensus on retrospective evaluation of large language model applications in clinical scenarios
Year: 2025
Type: article
Abstract: Large Language Models (LLMs), trained on vast amounts of textual data, have demonstrated strong capabilities in natural language understanding and generation. In the medical field, LLMs are increasingly applied across various domains such as disease screening, diagnostic assistance, and health management, playing a key role in advancing intelligent healthcare. In recent years, China has actively promoted the integration of artificial intelligence with healthcare through a series of policies that support enterprises in making breakthroughs in key technologies such as medical large language models and multi-modal data integration. Concurrently, efforts have accelerated the deployment of AI in applications such as health management and precision medicine, to gradually establish a full-cycle intelligent healthcare system encompassing prevention, diagnosis, treatment, and rehabilitation. However, the rapid deployment of LLMs in healthcare has highlighted the lack of standardized evaluation criteria and consistent methodologies. To address this, this expert consensus focuses on establishing a retrospective evaluation framework tailored to medical applications. By integrating scientific evaluation metrics, standards, and procedures, the framework provides clear and actionable guidance for model evaluators, developers, and end-users. It aims to unify assessment practices, enhance the scientific rigor and comparability of evaluations, and ensure the safe and effective use of LLMs in healthcare, ultimately supporting the high-quality development of AI-powered medical services.
Source: Intelligent Medicine
Authors: Qing Chang, Fei Chen, Yaolong Chen, Longlong Cheng, Di Dong, Jiahong Dong, Xiaobin Feng, Junbo Ge, Jingjing He, Yihua He, Zhiyang He, Hong Ji, Xue Jiang, Zehua Jiang, Nan Li, Peng Li, Yazi Li, Bing Liu, Junwei Liu, Han Lyu, Dong Min, Wen'an Qi, Xibin Shen, Bin Sheng, Jing Sun, Yujie Sun, Binglei Tian, Kailei Wang, Li Wang, Lihua Wang, Weimin Wang, Ye Wang, Yuqin Wang, Zhenchang Wang, Jianping Weng, Junmin Wei, Gang Wu, Xiaojian Wu, Yue Xiao, Yanwu Xu, Peng Yan, Zhewei Ye, Wanhong Yin, Chengwen Zhang, Di Zhang, Pengjun Zhang, Wensheng Zhang, Xudong Zhang, Shijie Zhao, Yi Zhao, Shaohua Zhou, Xiang Zhou, Baoliang Zhu, Lifeng Zhu, Zuyi Zhu
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $1,100
OpenAlex ID: W4415019705