Local Decoding and Update of Compressed Data

Abstract

This paper investigates data compression that simultaneously allows local decoding and local update. The main result is a universal compression scheme for memoryless sources with the following features. The rate can be made arbitrarily close to the entropy of the underlying source, contiguous fragments of the source can be recovered or updated by probing or modifying a number of codeword bits that is on average linear in the size of the fragment, and the overall encoding and decoding complexity is quasilinear in the blocklength of the source. In particular, the local decoding or update of a single message symbol can be performed by probing or modifying a constant number of codeword bits. This latter part improves over previous best known results for which local decodability or update efficiency grows logarithmically with blocklength.

O (log log n) Worst-Case Local Decoding and Update Efficiency for Data Compression

“…This can be generalized to the scenario where p X is unknown to the encoder and decoder by first estimating p X and then using this for designing the compression scheme as in[1]. Similarly, our results can be generalized to nonbinary alphabets.…”

“…For simplicity we assume here that p X is a known distribution over X " t0, 1u. 1 A fixed-length compression scheme of rate R consists of a pair of algorithms, the encoder ENC and the decoder DEC. The encoder maintains for every X n , a codeword C nR P t0, 1u nR such that DECpC nR q is a good estimate of X n .…”

“…It was recently shown in [1] that pd, uq " pOp1q, Op1qq is achievable. In that paper the authors also gave a separate compression scheme achieving 2 pd wc , uq " pOplog log nq, Oplog log nqq.…”

Local Editing in LZ-End Compressed Data

“…In addition, some purpose-built algorithms allow editing parts of the compressed data string [6]. Recent innovations allow adapting any universal compressor to allow random access [7], [8], and random access with local editing [8]. All of these have been achieved without decompressing the entire string, which processing compressed data traditionally requires.…”

“…The limitations of the algorithms of Tatwawadi et al [7] and Vatedka and Tchamkerten [8] are that neither method applies to the parsing algorithm directly. Both algorithms involve breaking the data into a series of blocks which are compressed independently of one another by an arbitrary parsing algorithm (e.g.…”