Computable Bounds for Rate Distortion With Feed Forward for Stationary and Ergodic Sources

Abstract

In this paper, we consider the rate distortion problem of discrete-time, ergodic, and stationary sources with feed forward at the receiver. We derive a sequence of achievable and computable rates that converge to the feed-forward rate distortion. We show that for ergodic and stationary sources, the rate is achievable for any , where the minimization is performed over the transition conditioning probability such that . We also show that the limit of exists and is the feed-forward rate distortion. We follow Gallager's proof where there is no feed forward and, with appropriate modification, obtain our result. We provide an algorithm for calculating using the alternating minimization procedure and present several numerical examples. We also present a dual form for the optimization of and transform it into a geometric programming problem.Index Terms-Alternating minimization procedure, Blahut-Arimoto (BA) algorithm, causal conditioning, concatenating code trees, directed information, ergodic and stationary sources, ergodic modes, geometric programming (GP), rate distortion with feed forward.

Analogy between gambling and measurement-based work extraction

“…It then follows from Lemma 1 that the maximization problem in (53) can be solved using the tools of convex optimization, if P is convex. Alternatively, the alternating maximization procedure can be used to maximize over each term P X i |X i−1 ,Y i−1 separately while setting all other terms to be constant, beginning with i = n and moving backward to i = 1, similarly to [27]. Since each term depends only on previous terms and not on the following ones, this procedure will yield the global maximum as needed.…”

An achievability proof for the lossy coding of Markov sources with feed-forward

“…This function was shown to be easily analytically evaluated for some special classes of sources in [6] and its numerical calculation for stationary and ergodic sources was addressed in [2]. A simpler formula for stationary and ergodic sources was obtained in [2] using the notion of nth order feed-forward rate-distortion function.…”

“…This function was shown to be easily analytically evaluated for some special classes of sources in [6] and its numerical calculation for stationary and ergodic sources was addressed in [2]. A simpler formula for stationary and ergodic sources was obtained in [2] using the notion of nth order feed-forward rate-distortion function. In [7], Weissman and El Gamal gave a simple, yet inspiring, scheme to achieve the rate-distortion function when both decoder and encoder know the SI (causally or non-causally) based on an appropriate partitioning of the source sequence X n before encoding.…”

“…In [7], Weissman and El Gamal gave a simple, yet inspiring, scheme to achieve the rate-distortion function when both decoder and encoder know the SI (causally or non-causally) based on an appropriate partitioning of the source sequence X n before encoding. The achievability schemes proposed in [2], [3] are based on codetrees.…”

“…In this work, we adopt the idea given in [7] and developed in [10] and propose a constructive coding scheme for mary Markov sources which achieves the feed-forward rate distortion function and is conceptually simpler than the one given in [2], [3]. This paper is organized as follows.…”