Auditing specialist advice integration for pleural effusion detection with a frozen vision language model
Year: 2026
Type: preprint
Abstract: Background: A specialist classifier can improve a vision language model (VLM) while the resulting system still fails to improve on the specialist itself. We audited these distinct comparisons for pleural effusion and evaluated a fixed response-map repair rule. Methods: We retained a frozen Qwen3-VL-4B-Instruct implementation, including a documented affine patch-projection modification, and two DenseNet121 specialists trained on CheXpert (A) and MIMIC-CXR (B). Image-only predictions and responses to both possible anonymous specialist labels defined complete binary response maps. We analyzed 352 ChestX-Det confirmation images and a separately locked OpenI cohort of 99 patients. The ChestX-Det comparison of VLM+B against B was exploratory and was defined after the confirmation analysis; the external comparison was specified before inference. Paired 95% intervals used 10,000 joint label-stratified bootstrap resamples. Results: On ChestX-Det, VLM+B achieved 73.81% balanced accuracy versus 70.17% for B, a difference of 3.64 percentage points (95% interval 1.85 to 5.69). It corrected 13 B errors and introduced none; all improvements occurred on constant-response maps. The downstream A-to-B replacement gain exceeded the standalone gain by 0.58 points (−1.72 to 2.88), leaving amplification unestablished. On OpenI, which contained six positive and 93 explicitly negative patients, VLM+B and B made identical predictions and both achieved 77.96% balanced accuracy. Relative to image-only, integration corrected two missed positives but introduced 39 false positives, a difference of −4.30 points (−22.58 to 15.59). The fixed repair was redundant for both possible supplied labels on every measured case. All 327 external responses were valid and all 30 repeats agreed. Conclusions: A modest internal integration advantage did not recur in the small external audit. The repair rule changed no decisions in either cohort. Unresolved internal patient dependence, report-derived external labels, and availability-related loss of positive cases preclude claims of general superiority, population equivalence, or clinical readiness.
Author: Mamun, S M Abdulla Al
Institution: University of Manchester
Language: English
Topic: COVID-19 diagnosis using AI
Subfield: Radiology, Nuclear Medicine and Imaging
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7236647138
