Searching for optimal alphabet for data compression using simulated annealing

Abstract

Data compression is very important today and it will be even more important in the future. Textual data use only limited alphabet -total number of used symbols (letters, numbers, diacritics, dots, spaces, etc.). In most languages, letters are joined into syllables and words. All three approaches are useful in text compression, but none of them is the best for any file. This paper describes a variant of algorithm for evolving alphabet from characters, 2-grams and 3-grams, which is optimal for compression of text files. We used Simulated Annealing for this evolution of the alphabet. The efficiency of the new variant will be tested on four compression algorithms. The achieved results are very promising.