An efficient compression scheme for data communication which uses a new family of self‐organizing binary search trees

Abstract

SUMMARYIn this paper, we demonstrate that we can effectively use the results from the field of adaptive selforganizing data structures in enhancing compression schemes. Unlike adaptive lists, which have already been used in compression, to the best of our knowledge, adaptive self-organizing trees have not been used in this regard. To achieve this, we introduce a new data structure, the partitioning binary search tree (PBST) which, although based on the well-known binary search tree (BST), also appropriately partitions the data elements into mutually exclusive sets. When used in conjunction with Fano encoding, the PBST leads to the so-called Fano binary search tree (FBST), which, indeed, incorporates the required Fano coding (nearly equal probability) property into the BST. We demonstrate how both the PBST and the FBST can be maintained adaptively and in a self-organizing manner. The updating procedure that converts a PBST into an FBST, and the corresponding new tree-based operators, namely the shift-to-left and the shift-to-right operators, are explicitly presented. The encoding and decoding procedures that also update the FBST have been implemented and rigorously tested. Our empirical results on the files of the well-known benchmarks, the Calgary and Canterbury Corpora, show that the adaptive Fano coding using FBSTs, the Huffman, and the greedy adaptive Fano coding achieve similar compression ratios. However, in terms of encoding/decoding speed, the new scheme is much faster than the latter two in the encoding phase, and they achieve approximately the same speed in the decoding phase. We believe that the same philosophy, namely that of using an adaptive self-organizing BST to maintain the frequencies, can also be utilized for other data encoding mechanisms, even as the Fenwick scheme has been used in arithmetic coding.

New Algorithms and Lower Bounds for Sequential-Access Data Compression

“…The rest of this chapter is based on joint work with Nekrich [GN] that shows how, on a RAM with Ω(log n)-bit words, we can speed up our algorithm even more, to both encode and decode in O(1) worst-case time. We note that Rueda and Oommen [RO04, RO06,RO08] have demonstrated the practicality of certain implementations of adaptive Fano coding, which is somewhat related to our algorithm ,especially a version [Gag04] in which we maintained an explicit code-tree.…”