Year: 2026
Type: preprint
Abstract: . Training machine learning (ML) emulators on km-scale Earth system models (ESMs) would require more data than is practical to store. Lossy data compression is a natural solution to this problem, but adoption by the ESM community has been slow, partly because of the fear of negatively impacting scientific conclusions. In this study, we demonstrate what level of lossy compression is acceptable for an ML-based emulator of the E3SMv3 atmospheric model at 1° resolution. We specifically use the ZFP compressor, which provides state-of-the-art performance and is integrated into NetCDF4. Thus, ZFP can be applied to the E3SM data output directly and the compressed data read easily with Python tools. We measure lossy compression impact by looking at the skill of the ACE2-emulator for AMIP-like runs. We find that a compression rate of roughly 5x does not affect the physics predictions of the ACE2-emulator. As we compress the data more aggressively, some metrics start to degrade, and at a compression rate of 11x many metrics have become unacceptable.
Language: English
Subfield: Atmospheric Science
Field: Earth and Planetary Sciences
Domain: Physical Sciences
SDG: Climate action
Open Access status: green
OpenAlex ID: W7225335510