Optimal Feedback Communication Via Posterior Matching

Abstract

In this paper we introduce a fundamental principle for optimal communication over general memoryless channels in the presence of noiseless feedback, termed posterior matching. Using this principle, we devise a (simple, sequential) generic feedback transmission scheme suitable for a large class of memoryless channels and input distributions, achieving any rate below the corresponding mutual information. This provides a unified framework for optimal feedback communication in which the Horstein scheme (BSC) and the Schalkwijk-Kailath scheme (AWGN channel) are special cases. Thus, as a corollary, we prove that the Horstein scheme indeed attains the BSC capacity, settling a longstanding conjecture. We further provide closed form expressions for the error probability of the scheme over a range of rates, and derive the achievable rates in a mismatch setting where the scheme is designed according to the wrong channel model. Several illustrative examples of the posterior matching scheme for specific channels are given, and the corresponding error probability expressions are evaluated. The proof techniques employed utilize novel relations between information rates and contraction properties of iterated function systems. . 1 The rate and error exponent analysis in the original papers [3, 6], while intuitively appealing, are widely considered to be non-rigorous. by U. A measurable bijective function µ : (0, 1) → (0, 1) is called a uniformity preserving function (u.p.f.) if Θ ∼ U implies that µ(Θ) ∼ U.A scalar distribution P X is said to be (strictly) dominated by another distribution P Y if F X (x) < F Y (x) whenever F Y (x) ∈ (0, 1), and the relation is denoted by P 0 for every A ∈ B, where B is the corresponding σ-algebra. This relation is denoted P X < < P Y . If both distributions are absolutely continuous w.r.t. each other, then they are said to be equivalent. The total variation distance between P X and P Y is defined asA statement is said to be satisfied for P X -a. a. (almost all) x, if the set of x's for which it is satisfied has probability one under P X .In what follows we use conv(·) for the convex hull operator, |∆| for the length of an interval ∆ ⊆ R, log for log 2 , range(f) for the range of a function f , and • for function composition. The indicator function over a set A is denoted by 1 A (·). A set A ⊆ R m is said to be convex in the direction u ∈ R m , if the intersection of A with any line parallel to u is a connected set (possibly empty). Note that A is convex if and only if it is convex in any direction.The following simple lemma states that (up to discreteness issues) any real-valued r.v. can be shaped into a uniform r.v. or vice versa, by applying the corresponding c.d.f or its inverse, respectively. This fact is found very useful in the sequel Lemma II.

Finite-Length Linear Schemes for Joint Source-Channel Coding over Gaussian Broadcast Channels with Feedback

“…Thus, at each channel use, only the "missing information" is transmitted. The work [13] generalized this idea and presented the posterior matching principle for optimal transmission over memoryless PtP channels with FB: At each channel symbol the transmitter should send only information that is independent of the past transmitted symbols, and is relevant for the reconstruction of the transmitted message.…”

“…Since minimizing K OL over all matrices F and over all covariance matrices Q w is rather involved, in the following we aim at setting ρ 0 = E{ 1,0 2,0} E{ 2 1,0 }E{ 2 2,0 } to be as large as possible. We next discuss two special instances of (13). To simplify the analytic treatment, we focus on the symmetric setting in which σ…”

LQG control approach for Gaussian MAC with feedback

“…By Lemma II.3 in [9], if condition (42) is satisfied then for all n, we can map the message set M n = {1, . .…”

The feedback capacity of the binary symmetric channel with a no-consecutive-ones input constraint

“…3 determines the input distribution and can be tracked by the encoder and the decoder. Having the instantaneous input distribution at both parties, one can apply a matching scheme [14], but conditioned on the previous input to the channel. This capacity-achieving scheme appears in [13].…”