The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions P B i |B i−1 ,A i : i = 0, 1, . . . , n and P B i |B i−1 i−M ,A i : i = 0, 1, . . . , n , where M is the memory of the channel, B n = {B j : j = . . . , 0, 1, . . . , n} are the channel outputs and A n = {A j : j = . . . , 0, 1, . . . , n} are the channel inputs. The characterizations of FTFI capacity, are obtained by first identifying the information structures of the optimal channel input conditional distributionsThe feedback capacity is supremum of all achievable rates, i.e., defined by C sup{R : R is achievable}.The underlying assumption for C FB A ∞ →B ∞ to correspond to feedback capacity is that the source process X i : i = 0, . . . , to be encoded and transmitted over the channel has finite entropy rate, and satisfies the following conditional independence [2].Coding theorems for channels with memory with and without feedback are developed extensively over the years, in an anthology of papers, such as, [3]-[15], in three direction. For jointly stationary ergodic processes, for information stable processes, and for arbitrary nonstationary and nonergodic processes. Since many of the coding theorems presented in the above references are either directly applicable or applicable subject to the assumptions imposed in these references, the main emphasis of the current investigation is on the characterizations of FTFI capacity, for different channels with transmission cost. 1 The subscript notation on probability distributions and expectation, i.e., P µ and E µ {·} is often omitted because it is clear from the context. 2 The superscript on expectation, i.e., P g indicates the dependence of the distribution on the encoding strategies.

This paper investigates applications of nonanticipative Rate Distortion Function (RDF) in a) zero-delay Joint Source-Channel Coding (JSCC) design based on average and excess distortion probability, b) in bounding the Optimal Performance Theoretically Attainable (OPTA) by noncausal and causal codes, and computing the Rate Loss (RL) of zero-delay and causal codes with respect to noncausal codes. These applications are described using two running examples, the Binary Symmetric Markov Source with parameter p, (BSMS(p)) and the multidimensional partially observed Gaussian-Markov source. For the multidimensional Gaussian-Markov source with square error distortion, the solution of the nonanticipative RDF is derived, its operational meaning using JSCC design via a noisy coding theorem is shown by providing the optimal encoding-decoding scheme over a vector Gaussian channel, and the RL of causal and zero-delay codes with respect to noncausal codes is computed. For the BSMS(p) with Hamming distortion, the solution of the nonanticipative RDF is derived, the RL of causal codes with respect to noncausal codes is computed, and an uncoded noisy coding theorem based on excess distortion probability is shown. The information nonanticipative RDF is shown to be equivalent to the nonanticipatory -entropy, which corresponds to the classical RDF with an additional causality or nonanticipative condition imposed on the optimal reproduction conditional distribution.

Abstract-We consider a unit memory channel, called Binary State Symmetric Channel (BSSC), in which the channel state is the modulo2 addition of the current channel input and the previous channel output. We derive closed form expressions for the capacity and corresponding channel input distribution, of this BSSC with and without feedback and transmission cost. We also show that the capacity of the BSSC is not increased by feedback, and it is achieved by a first order symmetric Markov process.

No abstract

A methodology is developed to realized optimal channel input conditional distributions, which maximize the finite-time horizon directed information, for channels with memory and feedback, by information lossless randomized strategies. The methodology is applied to general Time-Varying Multiple Input Multiple Output (MIMO) Gaussian Linear Channel Models (G-LCMs) with memory, subject to average transmission cost constraints of quadratic form. The realizations of optimal distributions by randomized strategies are shown to exhibit a decomposion into a deterministic part and a random part. The decomposition reveals the dual role of randomized strategies, to control the channel output process and to transmit new information over the channels. Moreover, a separation principle is shown between the computation of the optimal deterministic part and the random part of the randomized strategies.The dual role of randomized strategies generalizes the Linear-Quadratic-Gaussian (LQG) stochastic optimal control theory to directed information pay-offs.The characterizations of feedback capacity are obtained from the per unit time limits of finite-time horizon directed information, without imposingá priori assumptions, such as, stability of channel models or ergodicity of channel input and output processes. For time-invariant MIMO G-LCMs with memory, it is shown that whether feedback increases capacity, is directly related to the channel parameters and the transmission cost function, through the solutions of Riccati matrix equations, and moreover for unstable channels, feedback capacity is non-zero, provided the power exceeds a critical level.