Work

Diverse Concept Modeling (DiCoMX): Domain-specific self-supervised pre-training strategy for chest X-ray foundation models

Year: 2026

Type: article

Abstract: Foundation models trained on vast amounts of unlabeled data are transforming general-purpose learning, demonstrating generalizability across diverse domains. Such models are emerging as versatile and precise diagnostic tools in the medical field, signaling a transformative shift in clinical AI applications. In line with these developments, we introduce Diverse Concept Modeling (DiCoMX), a self-supervised learning (SSL) framework toward developing domain-specific chest X-ray (CXR) foundation models. DiCoMX is based on capturing concepts from CXR images using pseudo-labels through knowledge distillation from a teacher model. Our main idea is to use Vision Transformer (ViT) outputs to learn local pseudo-labels for individual tokens and a global pseudo-label for the entire CXR image. We pretrained a ViT on 600,638 CXRs and applied it to a range of independent downstream tasks, including pneumonia and COVID-19 detection in both adult and pediatric populations, multi label disease classification, and lung segmentation. Our experimental results demonstrate that DiCoMX outperforms existing SSL methods by learning more effective representations and improving model convergence. Therefore, DiCoMX learns transferable representations from large-scale unlabeled CXR datasets, providing a scalable step toward building domain-specific CXR foundation models. The code for pre-training foundation models using DiCoMX is publicly available at: https://github.com/Pediatric-Accelerated-Intelligence-Lab/DiCoMX .

Source: Biomedical Signal Processing and Control

DOI: 10.1016/j.bspc.2026.111135