Near-Optimal Rates for Limited-Delay Universal Lossy Source Coding

Abstract

International audienceWe consider the problem of limited-delay lossy coding of individual sequences. Here, the goal is to design (fixed-rate) compression schemes to minimize the normalized expected distortion redundancy relative to a reference class of coding schemes, measured as the difference between the average distortion of the algorithm and that of the best coding scheme in the reference class. In compressing a sequence of length T, the best schemes available in the literature achieve an O(T^-1/3) normalized distortion redundancy relative to finite reference classes of limited delay and limited memory, and the same redundancy is achievable, up to logarithmic factors, when the reference class is the set of scalar quantizers. It has also been shown that the distortion redundancy is at least of order T^-1/2 in the latter case, and the lower bound can easily be extended to sufficiently powerful (possibly finite) reference coding schemes. In this paper, we narrow the gap between the upper and lower bounds, and give a compression scheme whose normalized distortion redundancy is O(ln(T)/ T^1/2) relative to any finite class of reference schemes, only a logarithmic factor larger than the lower bound. The method is based on the recently introduced shrinking dartboard prediction algorithm, a variant of exponentially weighted average prediction. The algorithm is also extended to the problem of joint source-channel coding over a (known) stochastic noisy channel and to the case when side information is also available to the decoder (the Wyner–Ziv setting). The same improvements are obtained for these settings as in the case of a noiseless channel. Our method is also applied to the problem of zero-delay scalar quantization, where O(ln(T)/ T^1/2) normalized distortion redundancy is achieved relative to the (infinite) class of scalar quantizers of a given rate, almost achieving the known lower bound of order 1/ T^-1/2. The computationally efficient algorithms known for scalar quantization and the Wyner–Ziv setting carry over to our (improved) coding schemes presented in this paper

Efficient Tracking of Large Classes of Experts

“…The problem of limited-delay adaptive universal lossy source coding of individual sequences has recently been investigated in detail [18]- [20], [24], [39]- [41]. In the widely used model of fixedrate lossy source coding at rate R, an infinite sequence of [0, 1]-valued source symbols x 1 , x 2 , .…”

“…The relative loss with respect to the reference class Q is known in this context as the distortion redundancy. For the squared error distortion, the best randomized coding methods [20], [39], [41], with linear computational complexity with respect to the set Q, yield a distortion redundancy of order O(n −1/4 √ ln n). The problem of competing with the best time-variant quantizer that can change the employed quantizer several times (i.e., tracking the best quantizer), was analyzed in [24], based on a combination of [20] and the tracking algorithm of [4].…”

Random-Walk Perturbations for Online Combinatorial Optimization

“…In this situation, switching between routes might result in out-of-order delivery of packets due to changing delays, and eventually lead to decoding errors. Further examples of such problems include the online buffering problem described by Geulen et al [22] and the online lossy source coding problem of György and Neu [23]. A more abstract problem where the number of abrupt switches in the behavior is costly is the problem of online learning in Markovian decision processes, as described by Even-Dar et al [24] and Neu et al [25].…”

Prediction by Random-Walk Perturbation

“…In many applications, on would like to define forecasters that do not change their prediction too often. Examples of such problems include the online buffering problem described by Geulen, Voecking and Winkler [10] and the online lossy source coding problem of György and Neu [11]. A more abstract problem where the number of abrupt switches in the behavior is costly is the problem of online learning in Markovian decision processes, as described by Even-Dar, Kakade and Mansour [12] and Neu, György, Szepesvári and Antos [13].…”