Identifying Latent Reduced Models to Precondition Lossy Compression

Luo, Huizhang; Huang, Dan; Liu, Qing; Qiao, Zhenbo; Jiang, Hong; Bi, Jing; Yuan, Haitao; Zhou, MengChu; Wang, Jinzhen; Qin, Zhenlu

doi:10.1109/ipdps.2019.00039

Cited by 13 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With our proposed modeling and guideline for post-hoc analysis quality modeling, we can also accurately estimate the FFT quality degradation under high error bound situations. Figure 8 shows that the proposed estimation that considers error distribution from both Equations (10) and (11) outperforms previous solution that only considered uniform error distributions.…”

Section: Accuracy Of Post-hoc Analysis Quality Modelmentioning

confidence: 95%

“…1) PSNR: Figure 6 shows the measured PSNR compared to the estimated PSNR based on the error distribution. The dashed red line is the PSNR estimation based on the error distribution defined by Equation (10), which only considers the uniform distribution. The solid red line is the PSNR estimation that utilizes both Equations ( 10) and (11).…”

Section: Accuracy Of Post-hoc Analysis Quality Modelmentioning

confidence: 99%

See 1 more Smart Citation

Improving Prediction-Based Lossy Compression Dramatically via Ratio-Quality Modeling

Jin¹,

Di²,

Tian³

et al. 2021

Preprint

View full text Add to dashboard Cite

Error-bounded lossy compression is one of the most effective techniques for reducing scientific data sizes. However, the traditional trial-and-error approach used to configure lossy compressors for finding the optimal trade-off between reconstructed data quality and compression ratio is prohibitively expensive. To resolve this issue, we develop a general-purpose analytical ratio-quality model based on the prediction-based lossy compression framework, which can effectively foresee the reduced data quality and compression ratio, as well as the impact of lossy compressed data on post-hoc analysis quality. Our analytical model significantly improves the prediction-based lossy compression in three use-cases: (1) optimization of predictor by selecting the best-fit predictor; (2) memory compression with a target ratio; and (3) in-situ compression optimization by fine-grained tuning error-bounds for various data partitions. We evaluate our analytical model on 10 scientific datasets, demonstrating its high accuracy (93.47% accuracy on average) and low computational cost (up to 18.7× lower than the trialand-error approach) for estimating the compression ratio and the impact of lossy compression on post-hoc analysis quality. We also verify the high efficiency of our ratio-quality model using different applications across the three use-cases. In addition, our experiment demonstrates that our modeling based approach reduces the time to store the 3D RTM data with HDF5 by up to 3.4× with 128 CPU cores over the traditional solution.

show abstract

Section: Accuracy Of Post-hoc Analysis Quality Modelmentioning

confidence: 95%

Section: Accuracy Of Post-hoc Analysis Quality Modelmentioning

confidence: 99%

Improving Prediction-Based Lossy Compression Dramatically via Ratio-Quality Modeling

Jin¹,

Di²,

Tian³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Thus, error-bounded lossy compressors such as SZ [10,30,46], ZFP [31], and MGARD [1] have been developed to provide a much higher compression ratio while only introducing controllable distortion of data. Many prior studies have demonstrated the effectiveness of using those error-bounded lossy compressors for scientific data reduction [7,10,18,24,25,30,31,34,35,46,47] and improving I/O performance. [16,39,56].…”

Section: Introductionmentioning

confidence: 99%

Ceaz

Zhang

Jin

Geng

et al. 2022

Proceedings of the 36th ACM International Conference on Supercomputing

View full text Add to dashboard Cite

As HPC systems continue to grow to exascale, the amount of data that needs to be saved or transmitted is exploding. To this end, many previous works have studied using error-bounded lossy compressors to reduce the data size and improve the I/O performance. However, little work has been done for effectively offloading lossy compression onto FPGA-based SmartNICs to reduce the compression overhead. In this paper, we propose a hardware-algorithm codesign for an efficient and adaptive lossy compressor for scientific data on FPGAs (called CEAZ), which is the first lossy compressor that can achieve high compression ratios and throughputs simultaneously. Specifically, we propose an efficient Huffman coding approach that can adaptively update Huffman codewords online based on codewords generated offline, from a variety of representative scientific datasets. Moreover, we derive a theoretical analysis to support a precise control of compression ratio under an errorbounded compression mode, enabling accurate offline Huffman codewords generation. This also helps us create a fixed-ratio compression mode for consistent throughput. In addition, we develop an efficient compression pipeline by adopting cuSZ's dual-quantization algorithm to our hardware use cases. Finally, we evaluate CEAZ on five real-world datasets with both a single FPGA board and 128 nodes (to accelerate parallel I/O). Experiments show that CEAZ outperforms the second-best FPGA-based lossy compressor by 2.3× of throughput and 3.0× of ratio. It also improves MPI_File_write and MPI_Gather throughputs by up to 28.9× and 37.8×, respectively. CCS CONCEPTS• Computer systems organization → Reconfigurable computing.

show abstract