Report-Supervised Vision–Language Alignment for Label-Efficient Bronchoscopic Finding Recognition
Year: 2026
Type: article
Abstract: Background/Objectives: Fine-grained annotation of bronchoscopic findings is costly, whereas every bronchoscopic image is already described in the examination report. This single-centre proof-of-concept study investigated whether routine report text can serve as supervision for bronchoscopic finding recognition. Methods: BiomedCLIP was aligned with the image–caption pairs of the publicly available Bronchoscopy Examination Report Dataset (BERD; 4826 training images) using standard contrastive learning, a caption-aware objective that treats identical captions as positives, and a label-aware reference. The encoders were evaluated on the official test set, excluding three images duplicated from the training set (313 images, 180 patients), for zero-shot recognition of 23 report-derived finding categories, cross-modal retrieval, linear probing with 1–100% of the labelled training patients, and fine-tuning, using a hierarchical patient-level bootstrap and three training seeds. Results: Without training on finding labels, alignment raised zero-shot macro AUPRC from 0.184 to 0.255–0.278 and macro AUC-ROC from 0.59 to 0.77–0.78, and the gain held across ten prompt formulations. Retrieval generalised to captions never seen during training. With 1–25% of the labelled patients, linear probes on report-aligned features outperformed DINOv2, Swin-T and pretrained BiomedCLIP; with 5%, they exceeded DINOv2 and Swin-T trained with all labels. Caption-aware training showed no measurable benefit over standard contrastive training, full fine-tuning did not improve macro F1, and absolute performance remained modest (macro F1 at most 0.32). Conclusions: In this single-centre dataset, report text was an effective source of supervision for recognising report-derived finding categories with few labels; independent reference standards and external validation are needed before clinical use.
Source: Diagnostics
Author: İlkay Sibel Kervancı
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W7224440603
