Work

Histopathobiome – integrating histopathology and microbiome data via multimodal deep learning

Year: 2024

Type: article

Abstract: We introduce Histopathobiome, a term representing the integration of histopathology and microbiome data to explore tissue-microbe interactions. Using a dataset of colon biopsy whole-slide images paired with microbiota composition samples, we assess the benefits of combining these modalities to distinguish patients with inflammatory bowel disease (IBD) subtype – ulcerative colitis (UC) from non-IBD controls. Initially, we evaluate the unimodal performance of state-of-the-art algorithms using vectors representing bacterial species abundances or histopathology slide-level embeddings. We compare single-modality models with bimodal networks with various fusion strategies. Our results prove that histopathology and microbiome data are complementary in UC classification. By demonstrating improved performance over single-modality approaches, we prove that bimodal deep learning models can be used to learn meaningful and interpretable cross-modal tissue-microbe patterns.

Source: Zenodo (CERN European Organization for Nuclear Research)

DOI: 10.5281/zenodo.14039966

Authors: Polejowska, Agata, Annemarie, Boleij, Ciompi, Francesco

Institution: Radboud University Medical Center

Language: English

Topic: AI in cancer detection

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Good health and well-being

Open Access status: green

Funder: European Commission