Year: 2024
Type: preprint
Abstract: This paper presents error-bounded lossy compression tailored for particle datasets from diverse scientific applications in cosmology, fluid dynamics, and fusion energy sciences. As today's high-performance computing capabilities advance, these datasets often reach trillions of points, posing significant visualization, analysis, and storage challenges. While error-bounded lossy compression makes it possible to represent floating-point values with strict pointwise accuracy guarantees, the lack of correlations in particle data's storage ordering often limits the compression ratio. Inspired by quantization-encoding schemes in SZ lossy compressors, we dynamically determine the number of bits to encode particles of the dataset to increase the compression ratio. Specifically, we utilize a k-d tree to partition particles into subregions and generate ``bit boxes'' centered at particles for each subregion to encode their positions. These bit boxes ensure error control while reducing the bit count used for compression. We comprehensively evaluate our method against state-of-the-art compressors on cosmology, fluid dynamics, and fusion plasma datasets.
Source: arXiv (Cornell University)
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G102163711, G1193633358, Collaborative Research: OAC Core: Topology-Aware Data Compression for Scientific Analysis and Visualization, G1796307089, G2287611682, G262962599, III: Medium: Collaborative Research: Deep Learning for In Situ Analysis and Visualization, G3318903820, G3886122786, MULTIDIMENSIONAL PARAMETER-SPACE FEATURE TRACKING, ANALYSIS, AND VISUALIZATION, G498139845, G513797252, Collaborative Research: Frameworks: FZ: A fine-tunable cyberinfrastructure framework to streamline specialized lossy compression development, G5692331544, G6848031779, G6918803902, G878601127, G969889393, G993149817
OpenAlex ID: W4394006988