CoDE: A Patient Cohort-Based Study of Evidence Coverage and Support in Diagnostic Explanations Generated by Large Language Models
Year: 2026
Type: article
Abstract: Large language models (LLMs) generate medical diagnostic explanations using selected patient findings, raising the question of whether the cited evidence adequately supports the proposed diagnosis. Cohort models identify informative findings, but their value for evaluating independently generated explanations requires separate testing. We introduce CoDE (Cohort-based Diagnostic Evidence evaluation), a controlled empirical study comparing importance-weighted evidence coverage with joint diagnostic support from cited findings. On synthetic DDXPlus records and clinical-record-derived TRACE data, we examine evidence selection, generated explanations and controlled evidence changes using quantity-matched controls and separate evaluation predictors. Cohort importance improved selection, and guided evidence edits in DDXPlus changed diagnostic support more than matched random operations. For correct TRACE proposals, local positive-SHAP coverage exceeded uniform group coverage by 0.2371 failure-detection average precision (simultaneous 95% CI, 0.1093–0.3650), while tree-based direct support performed better. A second generator reproduced weighting gains, with uncertain ordering against direct support. Sparse failures and mapping sensitivity limited natural-explanation rankings in DDXPlus. These findings support using importance to prioritize evidence checks and joint support to assess evidence combinations. Potential omissions warrant testing by adding uncited evidence and measuring support changes; low coverage alone does not establish a reasoning error.
Source: Bioengineering
Authors: Ran Xin, Chengkai Wu, Yuxiang He, Jingsong Li, Yu Tian
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,258
OpenAlex ID: W7225650760
FWCI:
Cites:
Cited by:
Related to: