Work

Comparative performance of DeepSeek-R1 and DeepSeek-V3 on China’s National Medical Licensing Examination questions (2004–2022)

Year: 2026
Type: article
Abstract: BACKGROUND: DeepSeek, a cutting-edge artificial intelligence (AI) platform developed in China, has attracted growing interest for potential applications in clinical medicine and medical education. DeepSeek-V3 (MoE architecture) and DeepSeek-R1 (reinforcement learning-optimized for reasoning) are its flagship models, but the performance difference between DeepSeek‑R1 and DeepSeek‑V3 remains inconclusive. The objective of this study was to compare the performance of DeepSeek-R1 and DeepSeek-V3 on the Chinese National Medical Licensing Examination (NMLE). METHODS: This study employed DeepSeek-R1-0528 (released on May 28, 2025) and DeepSeek-V3-0324 (released on March 24, 2025) to perform the NMLE testing, and the subject area for the examination was Clinical Medicine. We collected the questions from NMLE 2004 through NMLE 2022, comprising 600 multiple-choice questions (MCQs) per annual examination, for a total of 11,400 MCQs. The NMLE tests were carried out using the free web-based version of DeepSeek, with DeepSeek-R1-0528 and DeepSeek-V3-0324 evaluated on separate computer systems. Model performance was assessed based on accuracy rates. RESULTS: The annual average accuracy rates for DeepSeek-R1-0528 and DeepSeek-V3-0324 were 91.2% (95% CI: 89.6% - 92.8%) and 91.1% (95% CI: 89.6% - 92.6%), respectively, with no significant difference observed between them (P = 0.591, Cohen's d = 0.086). The regression analysis showed that DeepSeek-R1-0528 showed no significant changes in accuracy rates from NMLE 2004 to NMLE 2013, while DeepSeek-V3-0324 showed a significant declining trend. Moreover, from NMLE 2014 to NMLE 2022, both models demonstrated significant declining trends in the accuracy rates. DeepSeek-R1-0528 and DeepSeek-V3-0324 answered 999 and 1,014 MCQs incorrectly, respectively, with 715 MCQs answered incorrectly by both models. CONCLUSIONS: DeepSeek-R1-0528 and DeepSeek-V3-0324 demonstrated near-identical and excellent performance on the Chinese NMLE.
Source: PLoS ONE
Language: English
Field: Medicine
Open Access status: gold
Estimated APC paid: $2,382
OpenAlex ID: W7236691391
FWCI:
Cites: 40
Cited by:
Related to: 10
Parts: 11