Foundation Models for Scientific Discovery: A Survey of Modalities, Architectures, and Pre-training Paradigms
Year: 2026
Type: preprint
Abstract: The integration of artificial intelligence with scientific discovery has entered a transformative era with the emergence of foundation models. These large-scale pre-trained models, characterized by their ability to transfer knowledge across diverse tasks and domains, are revolutionizing how we approach scientific problems in physics, chemistry, biology, and materials science. Moving beyond application-centric overviews, this survey offers a methodologically-driven analysis of foundation models in scientific discovery. We propose a systematic taxonomy organized along three orthogonal dimensions: data modality, model architecture, and pre-training paradigm. Through detailed analysis of representative models, we examine how these three dimensions interact to determine model capabilities, and identify effective combinations for different scientific tasks. We further discuss critical challenges including data scarcity, physical constraint integration, interpretability, and computational accessibility, with concrete technical recommendations. Finally, we outline actionable future directions toward more general, physics-informed, and interpretable foundation models for science. To facilitate ongoing research, we release an up-todate repository of resources at https://github.com/ Pinlong-Zhao/Scientific-Foundation-Models.
Author: Pinlong Zhao
Institution: Hangzhou Dianzi University
Language: English
Subfield: Materials Chemistry
Field: Materials Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7128314745