Year: 2025
Type: preprint
Abstract: Brain diseases, such as Alzheimer's disease and brain tumors, present profound challenges due to their complexity and societal impact. Recent advancements in brain foundation models have shown significant promise in addressing a range of brain-related tasks. However, current brain foundation models are limited by task and data homogeneity, restricted generalization beyond segmentation or classification, and inefficient adaptation to diverse clinical tasks. In this work, we propose SAM-Brain3D, a brain-specific foundation model trained on over 66,000 brain image-label pairs across 14 MRI sub-modalities, and Hypergraph Dynamic Adapter (HyDA), a lightweight adapter for efficient and effective downstream adaptation. SAM-Brain3D captures detailed brain-specific anatomical and modality priors for segmenting diverse brain targets and broader downstream tasks. HyDA leverages hypergraphs to fuse complementary multi-modal data and dynamically generate patient-specific convolutional kernels for multi-scale feature fusion and personalized patient-wise adaptation. Together, our framework excels across a broad spectrum of brain disease segmentation and classification tasks. Extensive experiments demonstrate that our method consistently outperforms existing state-of-the-art approaches, offering a new paradigm for brain disease analysis through multi-modal, multi-scale, and dynamic foundation modeling.
Source: arXiv (Cornell University)
Authors: Deng, Zhongying, Wang, Haoyu, Huang, Ziyan, Zhang, Lipei, Aviles-Rivero, Angelica I., Liu, Chaoyu, He, Junjun, Kourtzi, Zoe, Schönlieb, Carola-Bibiane
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: All in one cancer imaging optimisation using an integrated mathematical and deep learning approach, Nonlocal Methods for Arbitrary Data Sources, Robust and Efficient Analysis Approaches of Remote Imagery for Assessing Population and Forest Health in India, The Mathematics of Deep Learning, G434337789, Adaptive decision templates in the human brain, Cambridge Mathematics of Information in Healthcare (CMIH), EPSRC Centre for Mathematical and Statistical Analysis of Multimodal Clinical Imaging, Combining Knowledge And Data Driven Approaches to Inverse Imaging Problems, PET++: Improving Localisation, Diagnosis and Quantification in Clinical and Medical PET Imaging with Randomised Optimisation, Predicting Dementia: Optimising and translating AI to improve prognosis and clinical pathways
OpenAlex ID: W6979292479
Cites:
Cited by:
Related to: