Work

GMRNet: Deep Residual Network-Based Radiopathomic Glioma Classification with the Fusion of Multi-omics Data

Year: 2026

Type: article

Abstract: The classification of glioma subtypes is critical in clinical diagnosis and treatment planning. Multimodal deep learning enables more informative representations through the integration of diverse and complementary feature sources. In medical imaging, multimodal fusion of radiological and pathological data offers a practical approach for leveraging heterogeneous information for improved glioma subtype classification. Nevertheless, effectively extracting and integrating discriminative features from different modalities remains challenging. In this article, glioma multi-omics ResNet (GMRNet), which is a deep learning model for glioma subtype classification, is proposed. The model integrates radiomic and pathomic features through label-supervised semantic feature fusion to improve the classification performance. It uses a ResNet50 backbone to extract multiscale features from magnetic resonance imaging (MRI) data and whole-slide imaging (WSI) data separately and then fuses these features through feature concatenation. Multi-omics fusion increases the classification accuracy across glioma subtypes, including astrocytoma, oligodendroglioma, anaplastic astrocytoma, anaplastic oligodendroglioma, and glioblastoma. First, MRI and WSI data are passed through a systematic preprocessing pipeline to improve cross-sample consistency and preserve discriminative imaging structures. To overcome the limitations of scarce clinical data, diverse augmentation strategies, such as cropping, flipping, contrast adjustment, and affine transformations, are applied. Second, modality-specific features are extracted using two independent ResNet50-based branches. An MRI branch captures tumor morphology and structural patterns across multiple modalities, while a WSI branch focuses on cellular and tissue-level characteristics. This dual-branch design preserves the unique information of each modality and provides complementary perspectives for downstream fusion. Third, the extracted features are concatenated and passed through a fully connected bottleneck layer to integrate complementary cross-modal information and produce a compact fused representation. The fused representation is then fed into a multilayer classifier to predict glioma subtypes. Finally, the experimental results demonstrate that the proposed model achieves accuracies of 93.65% on the Huashan multimodal glioma (HMG) dataset and 86.47% on the open-access glioma (OAG) dataset. Compared with single-modality configurations, the multimodal fusion setting yields consistent empirical performance gains across both datasets.

Source: ACM Transactions on Multimedia Computing Communications and Applications

DOI: 10.1145/3812544

Authors: Hongwei Zeng, Fang Zhao, Lingchao Chen, Hongxia Xu, Honghao Gao, Bader Fahad Alkhamees