Year: 2026
Type: dissertation
Abstract: Deep Learning has achieved huge success in different fields lately, especially in the medical field; however, today’s traditional models are based on supervised learning and rely heavily on large amounts of expert-labeled data, which makes training time-consuming and a lot more expensive. This presents a significant challenge, especially for anomaly detection in medical images, where rare pathologies or unexpected findings are hard to find. For the workflow of Chest X-Ray (CXR) interpretation, the ability to find “out-of-distribution” anomalies without prior examples or a very limited number of ex-amples is critical for patient safety and efficiency in patient diagnosis. In this thesis I investigated the application of Vision-Language Models (VLMs) to per-form Zero-Shot and Few-Shot anomaly detection in Chest X-Rays. By using the joint em-bedding space of image and text representations (such as those found in CLIP or Bio-medCLIP), I propose a framework that identifies anomalies by measuring the semantic similarity between CXR images and the descriptive text prompts. I used different Prompt Engineering strategies ranging from simple labels to complex clinical descriptions to op-timize zero-shot performance. Subsequently, for few-shot adaptation, I used prototypical networks to compute class prototypes from the support embeddings without updating the model weights. Our result was inflated by patient-level data leakage, but after enforcing a strict patient-disjoint split, few-shot performance plateaued at 50-60% which indicates that prototypes captured patient identity rather than generalized pathology. In zero shot, BiomedCLIP achieved 55% balanced accuracy versus 41% for CLIP which demonstrates the benefits of medical pretraining while Qwen2-VL defaulted to predicting Normal, showing that just following instructions alone is not sufficient for medical anomaly detec-tion. This research shows that Vision-Language Models (VLMs) can be used for data-efficient and flexible medical AI systems. By reducing the reliance on large-scale, labeled datasets, these models can be more easily deployed for rare disease detection, and can assist radiol-ogists in further investigation and the identification of critical findings in resource con-strained environments. Keywords: Vision–Language Models, Anomaly Detection, Chest X-Ray, Zero-Shot Learning, Few-Shot Learning, Medical Imaging.
Author: Ashrafuzzaman, Md
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7198111354