Work

A multimodal framework to identify molecular mechanisms driving patient group-associated morphology through the integration of spatial transcriptomics and whole slide imaging

Year: 2026

Type: article

Abstract: Spatial organization of the disease microenvironment informs patient prognosis. Key modalities for studying spatial biology include H&E images (WSIs) for tissue structure and spatial transcriptomics (ST) for transcriptome-level programs. Spatial analysis aims to (1) identify markers linked to clinical outcome, (2) understand functional programs driving these associations, and (3) guide targeted therapies. Current research addresses these topics but offers limited explainability across the full morphology - molecular mechanism - outcome axis. Further, given the abundance of WSIs and limited availability of ST, there is a need for analyses integrating these complementary datasets. We present an AI-driven framework combining foundation-model features, multiple-instance learning, unsupervised clustering, and molecular analyses to identify mechanisms underlying outcome associated patterns. Applied to HER2+ breast cancer, we identify CCND1 and PTK6 signaling in tumor regions linked to trastuzumab resistance, consistent with prior studies. Our approach offers interpretable insights for multi-level resistance mechanisms, tissue-specific drug targeting, and precision medicine.

Source: npj Artificial Intelligence

DOI: 10.1038/s44387-025-00050-6

Authors: Reva Kulkarni, Avery Maddox, Sara Bailey, Arvind Rao

Institution: University of Michigan

Language: English

Topic: AI in cancer detection

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,590