The Improvised GZIP, A Technique for Real Time Lossless Data Compression

Abstract

Whenever it comes to data processing, the user always faces two major constraints. One is storage capacity and second is bandwidth. These two resources must be efficiently utilized by compressing the data. Enormous algorithms are used to compress data. As far as, compression in storage is concern, GZIP is used on large scale for lossless data compression. However, it is not desirable to carry out lossless data compression for real time data. In this paper, an improvisation is proposed in the existing GZIP algorithm for compressing real time data by a contemporary concept of introducing Adaptive Huffman algorithm by replacing the traditional Huffman encoder (static). Simulations have proved that improvised GZIP has approximate 18% better compression ratio and space saving than traditional GZIP for real time data. This research paper extends the usability of GZIP algorithm to carry out lossless compression for real time data.

A high-throughput big-data orchestration and processing system for the High Energy Photon Source

“…This requires the implementation of several key functionalities, including online compression, multi-level file assembly and parallel writing of multimodal data. MDW provides conventional compression methods such as Gzip (Shah & Sethi, 2019), Blosc (Alted, 2010) and LZ4 (Bartı ´k et al, 2015), and also plans to support advanced compression modes based on artificial intelligence. Data assembly in MDW is flexible and customizable, allowing users to define their own data structures at different beamlines.…”

Review of Data Science Trends and Issues in Porous Media Research With a Focus on Image‐Based Techniques

“…The ideal approach is to use the least number of bytes to store a dataset while not spending a considerable computational overhead in the read and write process (Barr & Asanović, 2006). There are many real‐time lossless data compression methods that can be used to reduce the computational resources while working with big data, such as GZIP, LZF, and SZIP (Shah & Sethi, 2019; Shen et al., 2019). These compression techniques can make a difference for storage of binary porous material images in which at least half of the voxels are zero.…”