Rényi divergence and the central limit theorem

Bobkov, Sergey G.; Chistyakov, G. P.; Götze, Friedrich

doi:10.1214/18-aop1261

Cited by 31 publications

(32 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This "triangular inequality"-like result exploits the nonnegativity of X, Y and captures the intrinsic cancellations of the 2 j terms of a Binomial expansion. If we do not have non-negativity, the standard expansion will provide a 2 j factor rather than 2 in the min{2, (e (∞) − 1) j } term (see e.g., Proposition 3.2 of Bobkov et al (2019)).…”

Section: Privacy Amplification For Rdpmentioning

confidence: 99%

Subsampled Rényi Differential Privacy and Analytical Moments Accountant

Wang

Balle²,

Kasiviswanathan

2020

JPC

157

242

View full text Add to dashboard Cite

We study the problem of subsampling in differential privacy (DP), a question that is the centerpiece behind many successful differentially private machine learning algorithms. Specifically, we provide a tight upper bound on the Renyi Differential Privacy (RDP) [Mironov, 2017] parameters for algorithms that: (1) subsample the dataset, and then (2) apply a randomized mechanism M to the subsample, in terms of the RDP parameters of M and the subsampling probability parameter.Our results generalize the moments accounting technique, developed by [Abadi et al. 2016] for the Gaussian mechanism, to any subsampled RDP mechanism.

show abstract

Section: Privacy Amplification For Rdpmentioning

confidence: 99%

Subsampled Rényi Differential Privacy and Analytical Moments Accountant

Wang

Balle²,

Kasiviswanathan

2020

JPC

157

242

View full text Add to dashboard Cite

show abstract

“…Wyner [8] and Yu and Tan [16], [20], [21] respectively used the KL divergence and the Rényi divergence to measure the level of approximation in the distributed source synthesis problem; Hayashi [3], [4] used the KL divergence to study the channel resolvability problem, and showed the optimal decay exponents of the KL divergence and the total variation are upper bounded by an expression involving the Rényi divergence. In probability theory, Barron [22] and Bobkov, Chistyakov and Götze [23] respectively used the KL divergence and the Rényi divergence to study the central limit theorem, i.e., they used them to measure the discrepancy between the induced distribution of sum of i.i.d. random variables and the normal distribution with the same mean and variance.…”

Section: Introductionmentioning

confidence: 99%

Rényi Resolvability and Its Applications to the Wiretap Channel

Tan

2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The conventional channel resolvability problem refers to the determination of the minimum rate required for an input process so that the output distribution approximates a target distribution in either the total variation distance or the relative entropy. In contrast to previous works, in this paper, we use the (normalized or unnormalized) Rényi divergence (with the Rényi parameter in [0, 2] ∪ {∞}) to measure the level of approximation. We also provide asymptotic expressions for normalized Rényi divergence when the Rényi parameter is larger than or equal to 1 as well as (lower and upper) bounds for the case when the same parameter is smaller than 1. We characterize the Rényi resolvability, which is defined as the minimum rate required to ensure that the Rényi divergence vanishes asymptotically. The Rényi resolvabilities are the same for both the normalized and unnormalized divergence cases. In addition, when the Rényi parameter smaller than 1, consistent with the traditional case where the Rényi parameter is equal to 1, the Rényi resolvability equals the minimum mutual information over all input distributions that induce the target output distribution. When the Rényi parameter is larger than 1 the Rényi resolvability is, in general, larger than the mutual information. The optimal Rényi divergence is proven to vanish at least exponentially fast for both of these two cases, as long as the code rate is larger than the Rényi resolvability. The optimal exponential rate of decay for i.i.d. random codes is also characterized exactly. We apply these results to the wiretap channel, and completely characterize the optimal tradeoff between the rates of the secret and non-secret messages when the leakage measure is given by the (unnormalized) Rényi divergence. This tradeoff differs from the conventional setting when the leakage is measured by the traditional mutual information.

show abstract

“…Additional assumptions are thus necessary. Nevertheless, for all 0 < p ≤ q, D p (µ||ν) ≤ D q (µ||ν), and similarly for T p (see [10]). Moreover, one clearly has…”

Section: Introductionmentioning

confidence: 88%