Search and Modification in Compressed Texts

Abstract

Text compression techniques like bzip2 lack the possibility to search or to update substrings at given positions of texts that have been compressed without prior decompression of the compressed text. We have developed Indexed Reversible Transformation (IRT), a modified version of the Burrows-Wheeler-Transformation (BWT) that in combination with run length encoding (RLE) and wavelet trees (WT) allows for position-based searching and updating substrings of compressed texts without prior decompression of the compressed text. As a result, IRT may be useful for a huge class of applications that due to space limitations prefer to search or to modify compressed texts instead of uncompressed texts.

Implementing Efficient Updates in Compressed Big Text Databases

“…Our contribution is based on a block-sorting technique called Indexed Reversible Transformation (IRT) [1] that modifies Burrows Wheeler Transformation (BWT) [2] in such a way that search, insertion, and deletion of the n th word are supported on the block-sorted text IRT(S). IRT(S) can then be compressed to CIRT (=compressed IRT) with run length encoding (RLE) and wavelet trees (WT).…”

“…IRT(S) can then be compressed to CIRT (=compressed IRT) with run length encoding (RLE) and wavelet trees (WT). Beyond our previous contribution [1], we present a technique called DICIRT (=Deletion and Insertion in CIRT) to directly insert the n th word into or to directly delete the n th word from a compressed text c(S) without full decompression of c(S), where c(S) is computed from S by CIRT.…”

“…In comparison to these approaches, we follow the idea of Indexed Reversible Transformation (IRT) [1], a self-index to block-sorted texts, which allows for position-based search, insert, update, and delete operations, e.g. of the n th word of a text, without full decompression of the text.…”

“…This paper proposes a technique called DICIRT that provides fast insert, update and delete operations on texts compressed by the Compressed Indexed Reversible Transformation (CIRT) [1], which already offers the following properties:…”

“…Beyond [1], DICIRT supports the following operations directly on the compressed text CIRT(S) without prior decompression of CIRT(S) to IRT(S) or to S and without time consuming bit-shift operations on compressed data:…”