Data compression and learning in time sequences analysis

Abstract

Motivated by the problem of the definition of a distance between two sequences of characters, we investigate the so-called learning process of typical sequential data compression schemes. We focus on the problem of how a compression algorithm optimizes its features at the interface between two different sequences A and B while zipping the sequence A + B obtained by simply appending B after A. We show the existence of a universal scaling function (the so-called learning function) which rules the way in which the compression algorithm learns a sequence B after having compressed a sequence A. In particular it turns out that it exists a crossover length for the sequence B, which depends on the relative entropy between A and B, below which the compression algorithm does not learn the sequence B (measuring in this way the relative entropy between A and B) and above which it starts learning B, i.e. optimizing the compression using the specific features of B. We check the scaling function on three main classes of systems: Bernoulli schemes, Markovian sequences and the symbolic dynamic generated by a non trivial chaotic system (the Lozi map). As a last application of the method we present the results of a recognition experiment, namely recognize which dynamical systems produced a given time sequence. We finally point out the potentiality of these results for segmentation purposes, i.e. the identification of homogeneous sub-sequences in heterogeneous sequences (with applications in various fields from genetic to time-series analysis).

Measuring complexity with zippers

“…In Figure 3 an experiment of recognition is reported [20]. Here an unknown sequence is compared, in the sense discussed in the previous section, with a number of known strings.…”

“…In this way the zipper "learns" the A file and, when encounters the B subsequence, tries to compress it with a coding optimized for A. If B is not too long [20,21], thus preventing LZ77 from learning it as well, the cross entropy per character can be estimated as:…”

Expanding the Algorithmic Information Theory Frame for Applications to Earth Observation

“…This term is intuitively close to C(x + ∆y) − C(x) in Equation (9), as both aim at expressing a small fraction of y only in terms of x. Secondly, the term C(y + ∆y) − C(y)in Equation (9) is intuitively close to C(∆y) in Equation (10), where in the former a representative dictionary extracted from y is used to code the fraction ∆y, while the latter discards any limitation regarding the size of the analysed objects and considers the full string y. This solves a problem raised in [30], which investigates the optimal size for ∆y in Equation (9), which does not represent y well enough if set too small, while it uses too much information from y itself in the compression step if set too large.…”

Artificial sequences and complexity measures

“…In [11] it has been studied in detail what happens when a compression algorithm tries to optimize its features at the interface between two different sequences A and B while zipping the sequence A + B obtained by simply appending B after A. It has been shown in particular the existence of a scaling function ruling the way the compression algorithm learns a sequence B after having compressed a sequence A.…”

“…A first field of activity [11,12] is that of segmentation problems, i.e. cases in which a unique string must be partitioned into subsequences according to some criteria to identify discontinuities in its statistical properties.…”