Recommendations for Comprehensive and Independent Evaluation of Machine Learning‐Based Earth System Models
Year: 2025
Type: article
Abstract: Machine learning (ML) is a revolutionary technology with demonstrable applications across multiple disciplines. Within the Earth science community, ML has been most visible for weather forecasting, producing forecasts that rival modern physics‐based models. Given the importance of deepening our understanding and improving predictions of the Earth system on all time scales, efforts are now underway to develop Earth‐system models (ESMs) capable of representing all components of the coupled Earth system (or their aggregated behavior) and their response to external changes over long timescales. Building trust in ESMs is a much more difficult problem than for weather forecast models, not least because the model must represent the alternate (e.g., future or paleoclimatic) coupled states of the system for which there are no direct observations. Given that the physical principles that enable predictions about the response of the Earth system are often not explicitly coded in these ML‐based models, demonstrating the credibility of ML‐based ESMs thus requires us to build evidence of their consistency with the physical system. To this end, this paper puts forward five recommendations to enhance comprehensive, standardized, and independent evaluation of ML‐based ESMs to strengthen their credibility and promote their wider use.
DOI: 10.1029/2024jh000496
Institutions: Lawrence Livermore National Laboratory, University of California, Davis, Colorado State University, Lawrence Berkeley National Laboratory, University of California, Berkeley, NSF National Center for Atmospheric Research, NSF NCAR Climate and Global Dynamics Laboratory, Indiana University Bloomington, Indiana University, Pacific Northwest National Laboratory, Oak Ridge National Laboratory, University of Maryland, College Park
Language: English
Subfield: Global and Planetary Change
Field: Environmental Science
Domain: Physical Sciences
SDG: Climate action
Open Access status: gold
Estimated APC paid: $2,500
Funders: National Science Foundation, U.S. Department of Energy, Battelle, Eli Lilly and Company, National Center for Atmospheric Research, Indiana University, Lilly Endowment, Global Down Syndrome Foundation, Office of Science, Environmental Resilience Institute, Indiana University, Biological and Environmental Research, Laboratory Directed Research and Development, Lawrence Livermore National Laboratory, Oak Ridge National Laboratory, Lawrence Berkeley National Laboratory, Pacific Northwest National Laboratory
Awards: Department of Energy (DoE) Support of the National Center for Atmospheric Research Activities, G107409864, G1079818582, G1088636562, G1143651532, G1202769797, G121915436, G125460123, G1286236842, G1337544421, G1356433355, G138090931, G14554683, G1645119126, G166562317, G1724155629, G1763799269, G1863003419, G1899721271, G203625172, G213222014, G2145270357, G223002938, G2270264153, The Management and Operation of the National Center for Atmoshperic Research (NCAR), G2411683797, G2506945562, G2510423872, G2757267231, G2812390658, G2820049545, G2981564522, G2988778103, G3083819904, G3121793570, G3137522415, G3351824818, G3571461640, G3669113783, G3861279505, G391136020, G3945850812, G3946874867, G3976335900, G4017386982, COOPERATIVE AGREEMENT TO ANALYZE VARIABILITY, CHANGE AND PREDICTABILITY IN THE EARTH SYSTEM (CATALYST), G419881185, G42916606, G4423001442, G4423657506, G4501827968, G450780475, G4744444288, G498139845, COOPERATIVE AGREEMENT TO ANALYZE VARIABILITY, CHANGE AND PREDICTABILITY IN THE EARTH SYSTEM (CATALYST), G5321382714, G5373841109, G5477578438, G5558627913, G5630301777, G5653134238, G5726405315, G574648691, G574795485, G5869080520, G6086734019, G613759197, G6486699926, G6511209046, INTERANNUAL AND MULTIDECADAL MODES OF CLIMATE VARIABILITY: MODAL EVOLUTIONS, LINKS, REGIONAL IMPACTS, AND E3SM REPRESENTATIONS, G660579501, G6652958823, G675320460, G7354628648, G7560550554, G7665161096, G7787287669, G8135867528, G8163035529, G821644656, G8269158468, G8326222658, G8420426323, G8906985441, G895790057, G969889393, G991281326
OpenAlex ID: W4408457719