It is well known that the rate-distortion function for a finite alphabet IID source with distribution p, denoted R(p, D), is uniformly continuous in its arguments. We prove an explicit bound on |R(p, D) − R(q, D)| for distributions p, q in terms of the variational distance p − q 1. A simple and elementary proof shows that |R(p,with constants depending on the distortion measure. The uniform continuity of the rate-distortion function has the same behavior as the uniform continuity of entropy in the order sense. The bounds are used for several applications. First, a simple sampling algorithm is presented to compute the rate-distortion function for an arbitrarily varying source to within a given accuracy. The uniform continuity bound is used here to roughly quantify the tradeoff between complexity and accuracy. Second, we comment on the problem of approximating the rate-distortion function for an unknown IID source to within a desired precision.

The issue of whether feedback can significantly increase reliability in the fixed-length channel code setting is further investigated. This paper considers the problem of error exponents for block codes with noiseless, delayed feedback used over discrete memoryless channels (DMCs) -including asymmetric channels with and without zeros in their transition matrix. We show that when output feedback is given to the encoder with a delay of T symbols, the error exponent is upper bounded by Esp(R − O((log T )/T )) + O((log T )/T ), where Esp denotes the sphere-packing exponent.

The purpose of this paper is to develop an information-theoretic understanding of the tradeoffs between decoder power, probability of error and decoding throughput. We start by considering the power consumed in the decoder circuit's interconnects, modeled as a lumped capacitor and resistor. After making simplifying assumptions about the decoder circuit, we use a sphere-packing technique to lower bound the decoding error probability for a given number of clock-cycles (or iterations). The analysis can be used to give lower bounds on probability of error versus total decoding power at a fixed decoding throughput.

Traditionally, coding has been seen as a way of sav ing transmit power: capacity-approaching codes require minimal transmitted energy-per-bit given the bandwidth available. But because transmit power is often smaller than decoding power at short distances, many recent wireless system designs continue to use uncoded transmission! We first observe that in wireless systems that both generate and face interference, coding serves another purpose (assuming interference is treated as noise): it allows a system to support a higher density of transmitter-receiver pairs. Bringing decoding power into the picture, we propose an approach to investigate which code/decoder to use and whether to use any coding at all. lt turns out that the code's gap to capacity determines how high the maximum supportable link density can be when power is plentiful, whereas the code's decoding complexity governs what link densities can be supported at low power.

Motivated by the lossy compression of an active-vision video stream, we consider the problem of finding the rate-distortion function of an arbitrarily varying source (AVS) composed of a finite number of subsources with known distributions. Berger's paper 'The Source Coding Game', IEEE Trans. Inform. Theory, 1971, solves this problem under the condition that the adversary is allowed only strictly causal access to the subsource realizations. We consider the case when the adversary has access to the subsource realizations non-causally. Using the type-covering lemma, this new rate-distortion function is determined to be the maximum of the IID rate-distortion function over a set of source distributions attainable by the adversary. We then extend the results to allow for partial or noisy observations of subsource realizations. We further explore the model by attempting to find the rate-distortion function when the adversary is actually helpful.Finally, a bound is developed on the uniform continuity of the IID rate-distortion function for finite-alphabet sources. The bound is used to give a sufficient number of distributions that need to be sampled to compute the rate-distortion function of an AVS to within a certain accuracy. The bound is also used to give a rate of convergence for the estimate of the rate-distortion function for an unknown IID finite-alphabet source .