Work

Generative versus discriminative diagnostic performance of large multimodal models in intracranial and spinal neuroradiology

Year: 2026

Type: article

Abstract: This study evaluated three large multimodal models (LMMs), GPT-5.2, Gemini 3 Pro, and Claude Opus 4.5, on 252 intracranial and spinal neuroradiology cases from the paywall-protected RSNA Case Collection using a two-stage paradigm comparing open-ended generative reasoning with multiple-choice discriminative recognition from a neurosurgical perspective. Generative primary diagnostic accuracy ranged from 51.6% to 62.7%, rising to 75.4–81.0% when answer options were provided, revealing a 15.1–23.8 percentage point gap. Among completely incorrect diagnoses, 43-59% were self-corrected upon provision of answer choices. Clinical impact grading by a neurosurgical panel demonstrated that clinically significant errors occurred in 17.8–26.9% of all cases. In a stratified 100-case subcohort with four models and repeated runs, and in a fresh-session re-evaluation, the same gap persisted and was not inflated by the within-session design. The low open-ended accuracy and the clinical error burden do not support autonomous diagnostic use; whether the stronger discriminative performance can support an assistive role will require prospective human-AI evaluation, and assessment should move beyond multiple-choice accuracy.

Source: npj Digital Medicine

DOI: 10.1038/s41746-026-03169-1