Entropy Reduction Using Context Transformations

Abstract

There are two classes of algorithms used in data compression. The first class deals directly with compression itself and it represents algorithms such as Huffman coding, Lempel-Ziv family algorithms, PPM and others. In the second class, there are algorithms trying to transform data into one, that are more easily compressed by the first class, for example Burrows-Wheeler transform or MoveToFront Coding. We prepared a second class method that transforms input data into data with lower entropy.Consider an input message, in which we know counts of all pairs of following symbols(bigrams). Suppose two bigrams αβ and αγ, the first bigram αβ doesn't occur in the message and a bigram αγ that occurs in the message. Since there is no occurence of αβ in the former message, we can replace each occurence of αγ for αβ and we note such a replacement. We know, that each time we encounter bigram αβ in the transformed message, it had to be created from αγ, so we are able to reconstruct the former message. Such a replacement we call a context transformation[1].If we add one more assumption to the above model, that probabilities of occurence of symbols β, γ follow p β > p γ , then the context transformation leads to increase of occurence of symbol β and decrease of occurence of symbol γ. If we chose context transformations in a convenient way, then we are able to decrease entropy of the former message and in consequence it leads to better compression attainable by entropy coder, such as Huffman coding.We tested the method on files from Calgary corpus and based on the file type, we were able to decrease entropy in the range 2-13%. There were file types, such as binary or image files, in which the method didn't work. Generally, we can state that using context transformations, we were able to manipulate with symbol's probability distribution to reach a better one that allows entropy coding algorithms to carry out more efficiently. References[1] M. Vasinek, "Kontextove mapy a jejich aplikace," Master's thesis, Vysoka skola banska -Technicka univerzita Ostrava, 2013. [Online]. Available: http: //hdl.handle.net/10084/98662

Prediction and Evaluation of Zero Order Entropy Changes in Grammar-Based Codes

“…Our recent studies were focused on a special class of grammar transforms that leave the message size intact [19,20]. In the present paper, the class of grammar transformations is extended with a novel concept of higher order context transformation [21].…”

“…The concept of context transformations was first proposed in [25], and the results were presented in [19]. It is the simplest transformation that assumes a pair of digrams beginning with the same symbol when one of the digrams is initially missing in the input message.…”

Higher Order Context Transformations

“…In our work we rely on this fact and we assume that Shannon's entropy of transformed data is approximately achievable. In our previous work about context transformations [2] and generalized context transformations [3] we studied, under what conditions the exchange of two different digrams, beginning with the same symbol, leads to the reduction of Shannon's entropy.…”

Generalized Context Transformations -- Enhanced Entropy Reduction

“…GCT is an improved version of the Context Transformation(CT) method presented recently [1]. GCT is used as a preprocessor for zero-order entropy coding algorithms like Arithmetic or Huffman coding.…”