Work

ClimSim: An open large-scale dataset for training high-resolution physics emulators in hybrid multi-scale climate simulators

Year: 2024
Type: article
Abstract: Modern climate projections lack adequate spatial and temporal resolution due to computational constraints, leading to inaccuracies in representing critical processes like thunderstorms that occur on the sub-resolution scale.Hybrid methods combining physics with machine learning (ML) offer faster, higher fidelity climate simulations by outsourcing compute-hungry, high-resolution simulations to ML emulators.However, these hybrid ML-physics simulations require domain-specific data and workflows that have been inaccessible to many ML experts.As an extension of the ClimSim dataset (Yu et al., 2024), we present ClimSim-Online, which also includes an end-to-end workflow for developing hybrid ML-physics simulators.The ClimSim dataset includes 5.7 billion pairs of multivariate input/output vectors, capturing the influence of high-resolution, high-fidelity physics on a host climate simulator's macro-scale state.The dataset is global and spans ten years at a high sampling frequency.We provide a cross-platform, containerized pipeline to integrate ML models into operational climate simulators for hybrid testing.We also implement various ML baselines, alongside a hybrid baseline simulator, to highlight the ML challenges of building stable, skillful emulators.The data (https://huggingface.co/datasets/LEAP/ClimSim_high-res 1 ) and code (https://leap-stc.github.io/ClimSimand https://github.com/leap-stc/climsim-online/) are publicly released to support the development of hybrid ML-physics and high-fidelity climate simulations.
Language: English
Open Access status: green
Awards: G1071216263, G1193633358, G1202769797, G122673668, G125460123, G1286236842, G1337544421, G1356433355, G138090931, G1453181110, G14554683, DISCOVERING PHYSICALLY MEANINGFUL STRUCTURES FROM CLIMATE EXTREME DATA, G1677143136, G1685430817, G1796307089, G1799333409, EXPLORING GLOBAL AEROSOL CLOUD INTERACTIONS IN UNPRECEDENTED NUMERICAL FIDELITY BY COMBINING BREAKTHROUGHS IN MULTI-SCALE MODELING, GPU SUPERCOMPUTING & NEURAL NETWORK PROCESS EMULATION., NRT-AI: Digital Transformation of Development (DToD), G1899721271, Track 4: Advanced CI Coordination Ecosystem: Monitoring and Measurement Services, STC: Center for Learning the Earth with Artificial Intelligence and Physics (LEAP), Collaborative Research: Advancing Understanding of Aerosol-Cloud Feedback Using the World's First Global Climate Model with Explicit Boundary Layer Turbulence, G2145270357, G217780377, G223002938, G2236757247, G2484515606, G2506945562, G2510423872, G2795275917, G2812390658, G2820049545, AGS-PRF: Physics-Constrained Machine Learning-Based Models for Climate Simulations with Data Assimilation and Uncertainty Quantification, G2865512842, G2900453181, DISCOVERING PHYSICALLY MEANINGFUL STRUCTURES FROM CLIMATE EXTREME DATA, G3083819904, RI: Small: Deep Variational Data Compression, G3562386289, G3780241159, G3861279505, G3886122786, G391136020, Track 1: ACCESS Resource Allocations Marketplace and Platform Services (RAMPS), G4010157413, G4094752214, G419881185, G42916606, DISCOVERING PHYSICALLY MEANINGFUL STRUCTURES FROM CLIMATE EXTREME DATA, G4501827968, ACO: An Open CI Ecosystem to Advance Scientific Discovery (OpenCI), G4932404565, G5076365615, Track 3: COre National Ecosystem for CyberinfrasTructure (CONECT), G5321382714, G5367127336, G5373841109, G5464919783, G5685548091, G5692331544, G574795485, G5793148246, G613759197, G6671297155, Understanding and Modelling the Earth System with Machine Learning, G7787287669, G8163035529, G821644656, Track 2: Customized Multi-tier Assistance, Training, and Computational Help (MATCH) for End User ACCESS to CI, G8326222658, G8506127299, State-dependent decadal predictability identified with explainable machine learning, G991281326
OpenAlex ID: W4401306841
FWCI: 1.314
Cites: 107
Cited by: 17
Related to: 9