Information Structures for Feedback Capacity of Channels With Memory and Transmission Cost: Stochastic Optimal Control and Variational Equalities

Abstract

The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions P B i |B i−1 ,A i : i = 0, 1, . . . , n and P B i |B i−1 i−M ,A i : i = 0, 1, . . . , n , where M is the memory of the channel, B n = {B j : j = . . . , 0, 1, . . . , n} are the channel outputs and A n = {A j : j = . . . , 0, 1, . . . , n} are the channel inputs. The characterizations of FTFI capacity, are obtained by first identifying the information structures of the optimal channel input conditional distributionsThe feedback capacity is supremum of all achievable rates, i.e., defined by C sup{R : R is achievable}.The underlying assumption for C FB A ∞ →B ∞ to correspond to feedback capacity is that the source process X i : i = 0, . . . , to be encoded and transmitted over the channel has finite entropy rate, and satisfies the following conditional independence [2].Coding theorems for channels with memory with and without feedback are developed extensively over the years, in an anthology of papers, such as, [3]-[15], in three direction. For jointly stationary ergodic processes, for information stable processes, and for arbitrary nonstationary and nonergodic processes. Since many of the coding theorems presented in the above references are either directly applicable or applicable subject to the assumptions imposed in these references, the main emphasis of the current investigation is on the characterizations of FTFI capacity, for different channels with transmission cost. 1 The subscript notation on probability distributions and expectation, i.e., P µ and E µ {·} is often omitted because it is clear from the context. 2 The superscript on expectation, i.e., P g indicates the dependence of the distribution on the encoding strategies.

Sequential Necessary and Sufficient Conditions for Capacity Achieving Distributions of Channels With Memory and Feedback

“…The starting point of our analysis is based on the information structures of the channel input conditional distribution developed in [11], and the convexity property of the extremum problem of feedback capacity derived in [10], [28] for abstract alphabet spaces and in [8] for finite alphabet spaces.…”

“…In this paper, we utilize recent work found in [10], [11], to develop such a methodology. Specifically, we derive sequential necessary and sufficient conditions for channel input distributions to maximize the finite horizon directed information.…”

Capacity Achieving Distributions and Separation Principle for Feedback Gaussian Channels With Memory: the LQG Theory of Directed Information

“…where the transmission cost functions are either one of the following two classes 1 Transmission Cost Functions Class A. γ i (T i a n , T i b n ) = γ A i (a i , b i ), (I.7)…”

“…In [1], it is shown that the maximization of I(A n → B n ) over all distributions P A i |A i−1 ,B i−1 : i = 0, . .…”

“…Stochastic optimal control theory and a variational equality of directed information are applied in [1], to identify the information structures of optimal channel input conditional distributions, P [0,n] = P A i |A i−1 ,B i−1 : i = 0, 1, . .…”