Capacity Upper Bounds for Deletion-type Channels

Abstract

We develop a systematic approach, based on convex programming and real analysis for obtaining upper bounds on the capacity of the binary deletion channel and, more generally, channels with i.i.d. insertions and deletions. Other than the classical deletion channel, we give special attention to the Poisson-repeat channel introduced by Mitzenmacher and Drinea (IEEE Transactions on Information Theory, 2006). Our framework can be applied to obtain capacity upper bounds for any repetition distribution (the deletion and Poisson-repeat channels corresponding to the special cases of Bernoulli and Poisson distributions). Our techniques essentially reduce the task of proving capacity upper bounds to maximizing a univariate, real-valued, and often concave function over a bounded interval. The corresponding univariate function is carefully designed according to the underlying distribution of repetitions, and the choices vary depending on the desired strength of the upper bounds as well as the desired simplicity of the function (e.g., being only efficiently computable versus having an explicit closed-form expression in terms of elementary, or common special, functions). Among our results, we show the following: (1) The capacity of the binary deletion channel with deletion probability d is at most (1 − d) log φ for d ≥ 1/2 and, assuming that the capacity function is convex, is at most 1 − d log(4/φ) for d < 1/2, where φ = (1 + √ 5)/2 is the golden ratio. This is the first nontrivial capacity upper bound for any value of d outside the limiting case d → 0 that is fully explicit and proved without computer assistance. (2) We derive the first set of capacity upper bounds for the Poisson-repeat channel. Our results uncover further striking connections between this channel and the deletion channel and suggest, somewhat counter-intuitively, that the Poisson-repeat channel is actually analytically simpler than the deletion channel and may be of key importance to a complete understanding of the deletion channel. (3) We derive several novel upper bounds on the capacity of the deletion channel. All upper bounds are maximums of efficiently computable, and concave, univariate real functions over a bounded domain. In turn, we upper bound these functions in terms of explicit elementary and standard special functions, whose maximums can be found even more efficiently (and sometimes analytically, for example, for d = 1/2). CCS Concepts: • Mathematics of computing → Information theory; • Theory of computation → Error-correcting codes;

Improved Capacity Upper Bounds for the Discrete-Time Poisson Channel

“…As with most previous capacity upper bounds for the DTP channel, we derive Theorem 1 with the help of a general convex duality framework, which we state below in a specialized form for the DTP channel. This framework was originally derived in [22] and has also been used to derive the state-of-the-art upper bound on C(µ) [4]. As discussed in [4], it is equivalent to other existing frameworks (e.g., see [23,7]).…”

“…From previous applications of Lemma 3 [22,4,24], it is apparent that designing candidate distributions Y so that the associated KL-gap is as small as possible leads to sharper capacity upper bounds. We follow this approach in this work as well.…”

“…To give some context, we start by presenting the family of digamma distributions originally introduced in [22] in order to derive capacity upper bounds for channels with synchronization errors, and which was also used to derive the state-of-the-art upper bounds on C(µ) in [4]. This is a family of distributions Y (q) parameterized by q ∈ (0, 1), and each distribution Y (q) satisfies Y (q) (y) = y 0 q y e g(y)−y y!…”

“…To obtain Theorem 1 from (18), we upper bound the term − ln α by an easy-to-compute expression in terms of λ and q, and then choose q appropriately as a function of λ and µ. From [22], we have the following upper bound on 1/y 0 , where y 0 is the normalizing factor of the digamma distribution Y (q) .…”

A Security Enhanced Encryption Scheme and Evaluation of Its Cryptographic Security

“…As reported in [24], the capacity Cap(d) of a deletion channel with the deletion rate d is upperbounded as follows:…”