SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors

Liang, Xin; Zhao, Kai; Di, Sheng; Li, Sihuan; Underwood, Robert; Gok, Ali Murat; Tian, Jiannan; Deng, Junjing; Calhoun, Jon C.; Tao, Dingwen; Cappello, Franck

doi:10.1109/tbdata.2022.3201176

Cited by 51 publications

(14 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The test platform is equipped with two 28-core Intel Xeon Gold 6238R processors and 384 GB of memory. We use SZ3 [40], which is easy to (de)couple lossless encoding from SZ compression, due to its high modularity. Comparison baselines.…”

Section: Methodsmentioning

confidence: 99%

TAC+: Drastically Optimizing Error-Bounded Lossy Compression for 3D AMR Simulations

Daoce¹,

Pulido²,

Grosset³

et al. 2023

Preprint

View full text Add to dashboard Cite

Today's scientific simulations require a significant reduction of data volume because of extremely large amounts of data they produce and the limited I/O bandwidth and storage space. Error-bounded lossy compression has been considered one of the most effective solutions to the above problem. However, little work has been done to improve error-bounded lossy compression for Adaptive Mesh Refinement (AMR) simulation data. Unlike the previous work that only leverages 1D compression, in this work, we propose an approach (TAC) to leverage high-dimensional SZ compression for each refinement level of AMR data. To remove the data redundancy across different levels, we propose several pre-process strategies and adaptively use them based on the data characteristics. We further optimize TAC to TAC+ by improving the lossless encoding stage of SZ compression to efficiently handle many small AMR data blocks after the pre-processing. Experiments on 8 AMR datasets from a real-world large-scale AMR simulation demonstrate that TAC+ can improve the compression ratio by up to 4.9× under the same data distortion, compared to the state-of-the-art method. In addition, we leverage the flexibility of our approach to tune the error bound for each level, which achieves much lower data distortion on two application-specific metrics.! 1. The patch-based AMR data redundantly saves the data block to be refined at the next finer level in the current coarse level (will be introduced in detail in Section 2.3).

show abstract

Section: Methodsmentioning

confidence: 99%

TAC+: Drastically Optimizing Error-Bounded Lossy Compression for 3D AMR Simulations

Daoce¹,

Pulido²,

Grosset³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…On the other hand, for the compression task, we compare our methods to the state-of-the-art compression scheme without neural network (Liang et al, 2022) and the recently proposed coordinate-based neural network for compression (Huang & Hoefler, 2023).…”

Section: Methodsmentioning

confidence: 99%

“…To this end, we combine our method with the existing HNeR-based compression scheme (Ladune et al, 2023) to apply image-wise compression, i.e., we train one neural network for the compression of each image. As a non-machine learning baseline, we consider a modular composable SZ3 (Liang et al, 2022) framework. We also compare with the Fourier feature network with temporal encoding (Huang & Hoefler, 2023, FFN-T) proposed for compression of climate and weather data.…”

Section: Weather and Climate Datamentioning

confidence: 99%

Relationship of Neural Correlates of Gait Characteristics and Cognitive Dysfunction in Patients With Mild Cognitive Impairment

Kim¹,

Park²,

Choi³

et al. 2023

Preprint

View full text Add to dashboard Cite

Background Some patients with mild cognitive impairment (MCI) experience gait disturbances. However, there are few reports on the relationship between gait disturbance and cognitive function in patients with MCI. Therefore, we investigated the neural correlates of gait characteristics related to cognitive dysfunction. Methods Eighty patients diagnosed with MCI from three dementia centres in Gangwon-do, Korea were recruited for this study. We defined MCI as a Clinical Dementia Rating global score of 0.5 or higher, with a memory domain score of 0.5 or greater. The patients were classified as having either early or late MCI based on their Mini Mental Status Examination z-scores. Multiple logistic regression analysis was performed to examine the association between the gait characteristics and cognitive impairment. Analyses included variables such as age, sex, years of education, number of comorbidities, body mass index and height. Results Gait velocity, step count, step length, heel-to-heel base support, swing and stance phase duration, and support time were associated with cognitive function. A decrease in gray matter volume in the right pericalcarine area was associated with gait characteristics related to cognitive dysfunction. An increase in the curvature of gray matter in the right entorhinal, right lateral orbitofrontal, right cuneus, and right and left pars opercularis areas was also associated with gait characteristics related to cognitive dysfunction. Conclusion Since gait impairment is an important factor in determining activities of daily living in patients with mild cognitive impairment, the evaluation of gait and cognitive functions in patients with mild cognitive impairment is important.

show abstract

“…Though data reduction is not the only intended use case for our approach, nor is it something we design for specifically (such as with network weight quantization for further data reduction [16]), we believe it is useful to the community to compare the compression ability of state-of-the-art SRNs with state-of-the-art compressors. We compare compression results from TTHRESH [2] and SZ3 [12,13,34] with our approach in Figure 5. We do not compare to a rendering-focused compressor such as cudaCompress [28] or a bricked version of TTHRESH [2], as this has been done by recent work [30] showing that its decoding is significantly slower, compression rates smaller, memory use is higher, and image quality is worse than state-of-the-art SNRs.…”

Section: Scene Representation Network Comparisonmentioning

confidence: 99%

Deep Hierarchical Super Resolution for Scientific Data

Wurster

Guo

Shen

et al. 2023

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Scene representation networks (SRNs) have been recently proposed for compression and visualization of scientific data. However, state-of-the-art SRNs do not adapt the allocation of available network parameters to the complex features found in scientific data, leading to a loss in reconstruction quality. We address this shortcoming with an adaptively placed multi-grid SRN (APMGSRN) and propose a domain decomposition training and inference technique for accelerated parallel training on multi-GPU systems. We also release an open-source neural volume rendering application that allows plug-and-play rendering with any PyTorch-based SRN. Our proposed APMGSRN architecture uses multiple spatially adaptive feature grids that learn where to be placed within the domain to dynamically allocate more neural network resources where error is high in the volume, improving state-of-the-art reconstruction accuracy of SRNs for scientific data without requiring expensive octree refining, pruning, and traversal like previous adaptive models. In our domain decomposition approach for representing large-scale data, we train an set of APMGSRNs in parallel on separate bricks of the volume to reduce training time while avoiding overhead necessary for an out-of-core solution for volumes too large to fit in GPU memory. After training, the lightweight SRNs are used for realtime neural volume rendering in our open-source renderer, where arbitrary view angles and transfer functions can be explored. A copy of this paper, all code, all models used in our experiments, and all supplemental materials and videos are available at https://github.com/skywolf829/APMGSRN.

show abstract

SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors

Cited by 51 publications

References 41 publications

TAC+: Drastically Optimizing Error-Bounded Lossy Compression for 3D AMR Simulations

TAC+: Drastically Optimizing Error-Bounded Lossy Compression for 3D AMR Simulations

Relationship of Neural Correlates of Gait Characteristics and Cognitive Dysfunction in Patients With Mild Cognitive Impairment

Deep Hierarchical Super Resolution for Scientific Data

Contact Info

Product

Resources

About