Work

A Survey on Medical Competence Evaluation Benchmarks for Large Language Models

Year: 2026

Type: article

Abstract: Large language models (LLMs) show considerable potential to revolutionize healthcare through their performance across diverse clinical applications. Given the inherent constraints of LLMs and the critical nature of medical practice, a rigorous and systematic evaluation of their medical competence is imperative. This study presents a comprehensive review of the established methodologies and benchmarks for evaluating the medical competence of LLMs, encompassing a thorough analysis of current assessment practices across medical knowledge, clinical practice competence, and ethical-safety considerations. By integrating clinician competency assessment frameworks into LLMs evaluation, we propose a structured tri-dimensional framework that systematically organizes existing evaluation approaches according to medical theoretical knowledge, clinical practice ability, and ethical-safety considerations. Furthermore, this research provides critical insights into future developmental trajectories while establishing foundational frameworks and standardization protocols for the integration of LLMs into medical practice.

Source: Health care science

DOI: 10.1002/hcs2.70050

Authors: Qiting Wang, Huiru Zou, Haobin Zhang, Yongshun Huang, Junzhang Tian, Weibin Cheng

Institutions: Guangdong Pharmaceutical University, Jinan University, Guangdong Provincial Occupational Disease Prevention Hospital, City University of Macau

Language: English

Topic: Artificial Intelligence in Healthcare and Education

Subfield: Health Informatics

Field: Medicine

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,300