Challenges and Recent Advances in Distributed Predictive Coding

Abstract

Abstract-Distributed coding of correlated sources with memory poses a number of considerable challenges that threaten practical applications, particularly (but not only) in the context of sensor networks. This problem is strongly motivated by the obvious observation that most common sources exhibit temporal correlations that may be at least as important as spatial or inter-source correlations. This paper presents an analysis of the underlying tradeoffs, paradigms for coding systems, and approaches for distributed predictive coder design optimization. Motivated by practical limitations on both complexity and delay (especially for dense sensor networks) the focus here is on predictive coding. From the source coding perspective, the most basic tradeoff (and difficulty) is due to conflicts that arise between distributed coding and prediction, wherein 'standard' distributed quantization of the prediction errors, if coupled with imposition of zero decoder drift, would drastically compromise the predictor performance and hence the ability to exploit temporal correlations. Another challenge arises from instabilities in the design of closed loop predictors, whose impact has been observed in the past, but is greatly exacerbated in the case of distributed coding. The main contribution focuses on the tradeoffs encountered within a more general paradigm where decoder drift is allowable or unavoidable, and must be effectively accounted for and controlled. We briefly review our earlier results on which we build to derive an overall design optimization method that avoids the pitfalls of naive distributed predictive quantization and produces an optimized low complexity and low delay coding system.

Distributed Predictive Coding for Spatio-Temporally Correlated Sources

“…: Entry , and is obtained as (20) 2) Loop Codebook : Entry , and is obtained as (21), shown at the bottom of the page, where is shorthand notation for .…”

“…However in both the previous settings, neither the optimality of the algorithms was proven nor the system can be guaranteed to be drift-free for all values of intersource/temporal correlations. We have proposed optimal algorithms with "zero-drift" and "controlled-drift" for distributed predictive coding in [20] and [21]. The "controlled-drift" algorithm includes the zero-drift approach as a special case that emerges whenever the impact of potential drift overwhelms the benefits of improved prediction.…”

Delayed Sequential Coding of Correlated Sources

“…Differential predictive coded modulation (DPCM) is a popular and well-established sequential predictive source compression method with a long history of development (see [1]- [8] and the references therein). DPCM has had wide impact on the evolution of compression standards for speech, image, audio, and video coding.…”