Meta-varlık güdümlü üçlü madenciliği ile tıbbi görüntü-dil modellerinin ince taneli hizalanması
Year: 2026
Type: dissertation
Abstract: Imaging-based diagnosis relies on the joint interpretation of medical images and radiology reports, yet growing data volumes strain clinical experts and increase the risk of error.Medical vision-language models (med-VLMs) address this by learning a shared representation of chest X-ray (CXR) images and their reports, and their effectiveness hinges on the quality of image-text alignment.Existing alignment methods are dominated by contrastive learning, which separates disease classes at a coarse level and overlooks the fine-grained pathology attributes-such as location, size, and severity-that distinguish clinically different studies within a class, yielding suboptimal representations.This thesis introduces MedTrim (Meta-entity-driven Triplet mining), an alignment method that improves precision through structured triplet learning guided by meta-entities extracted from radiology reports.Unlike contrastive and conventional triplet frameworks that rely on global class labels or implicit similarity references, MedTrim explicitly models the hierarchical relationships among pathology attributes to preserve clinically meaningful intra-class variation.To this end, it employs an ontology-based entity recognition (OBER) module to identify disease classes together with adjectival qualifiers and directional descriptors, a novel entity-aware score function that captures graded inter-sample similarity, and a multimodal triplet alignment objective that enforces consistency across image-report pairs sharing detailed pathology attributes without compromising within-modal relationships.Evaluated on public datasets, MedTrim improves performance on downstream retrieval, disease classification, and report generation over leading alignment methods, with the largest gains on fine-grained attributes and low-prevalence disease classes.
Author: Yılmaz, Melih Berk
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7213476311
