Achieving the Empirical Capacity Using Feedback: Memoryless Additive Models

Abstract

We address the problem of universal communications over an unknown channel with an instantaneous noiseless feedback, and show how rates corresponding to the empirical behavior of the channel can be attained, although no rate can be guaranteed in advance. First, we consider a discrete moduloadditive channel with alphabet X , where the noise sequence Z n is arbitrary and unknown and may causally depend on the transmitted and received sequences and on the encoder's message, possibly in an adversarial fashion. Although the classical capacity of this channel is zero, we show that rates approaching the empirical capacity log |X | − Hemp(Z n ) can be universally attained, where Hemp(Z n ) is the empirical entropy of Z n . For the more general setting where the channel can map its input to an output in an arbitrary unknown fashion subject only to causality, we model the empirical channel actions as the modulo-addition of a realized noise sequence, and show that the same result applies if common randomness is available. The results are proved constructively, by providing a simple sequential transmission scheme approaching the empirical capacity. In part II of this work we demonstrate how even higher rates can be attained by using more elaborate models for channel actions, and by utilizing possible empirical dependencies in its behavior.

Zero-Rate Feedback Can Achieve the Empirical Capacity

“…The first is the model considered in [3]-a binary modulo-additive channel with a noise sequence whose empirical frequency of 's is unknown. In this example, the "empirical mutual information" under all state sequences is maximized by the uniform distribution, so our algorithm achieves the "empirical capacity."…”

“…This rate can be thought of as the empirical mutual information of the channel with a uniform input distribution. Since the uniform input distribution achieves the capacity for all BSCs, this rate can also be called the empirical capacity, as in the work of Shayevitz and Feder [3].…”

“…One such benchmark that we consider in this paper is the empirical capacity-for a fixed , the empirical capacity is defined as the supremum over all input distributions of the empirical mutual information First used by Shayevitz and Feder [3], empirical capacity is given its name not because it is purported to be optimal, but instead because of its resemblance to the capacity of point-topoint discrete memoryless channels.…”

“…Then given any and channel input distribution , there exists an sufficiently large and an coding strategy with feedback rate (3) such that for all , the rate (4) is achievable with probability . 1 A question then arises of how one chooses the input distribution P .…”

Variable-Rate Channel Capacity

“…When the channel state is known at the transmitter, the average of the capacities achievable for the individual channel states is an important fundamental limit. Furthermore, the benefits of variable-length channel coding are well known when feedback is available (e.g., [4], [28], [32], and [22]). …”