Differential Privacy as a Mutual Information Constraint

Abstract

Differential privacy is a precise mathematical constraint meant to ensure privacy of individual pieces of information in a database even while queries are being answered about the aggregate. Intuitively, one must come to terms with what differential privacy does and does not guarantee. For example, the definition prevents a strong adversary who knows all but one entry in the database from further inferring about the last one. This strong adversary assumption can be overlooked, resulting in misinterpretation of the privacy guarantee of differential privacy. Herein we give an equivalent definition of privacy using mutual information that makes plain some of the subtleties of differential privacy. The mutual-information differential privacy is in fact sandwiched between $\epsilon$-differential privacy and $(\epsilon,\delta)$-differential privacy in terms of its strength. In contrast to previous works using unconditional mutual information, differential privacy is fundamentally related to conditional mutual information, accompanied by a maximization over the database distribution. The conceptual advantage of using mutual information, aside from yielding a simpler and more intuitive definition of differential privacy, is that its properties are well understood. Several properties of differential privacy are easily verified for the mutual information alternative, such as composition theorems.Comment: ACM CCS 2016, 12 page

Local Distribution Obfuscation via Probability Coupling

“…Since the seminal work of Dwork [1] on differential privacy (DP), a lot of its variants have been studied to provide different types of privacy guarantees [21]; e.g., d-privacy [13], f -divergence privacy [20], [8], mutualinformation DP [9], concentrated DP [22], Rényi DP [10], Pufferfish privacy [23], Bayesian DP [24], local DP [2], personalized DP [25], and utility-optimized local DP [26]. All of these are intended to protect single input values instead of input distributions.…”

“…In this paper, we relax the notion of DistP by generalizing it to an arbitrary divergence. The basic idea is similar to point privacy notions that relax DP and improve utility by relying on some divergence (e.g., total variation privacy [8], Kullback-Leibler divergence privacy [8], [9], and Rényi differential privacy [10]). We define the notion of divergence distribution privacy by replacing the DP-style with an arbitrary divergence D. This relaxation allows us to formalize "on-average" DistP, and to explore privacy notions *This work was partially supported by JSPS KAKENHI Grant JP17K12667, JP19H04113, and Inria LOGIS project.…”

Privacy-Utility Trade-off of Linear Regression under Random Projections and Additive Noise

“…Proposition 1 (Theorem 1 in [4]). ε-MI-DP is stronger than (ε, δ )-DP in the sense that for all ε > 0 if a mechanism is ε-MI-DP, there exists ε ′ , δ ′ such that the mechanism satisfies (ε ′ , δ ′ )-DP.…”

“…Proposition 2 (See Lemma 2 in [4]). If a mechanism is ε-MI-DP then it also satisfies (0, 2 log(e) ε)-DP.…”

“…This connects to the natural information-theoretic notion of privacy, as well as enabling the use of more standard tools for analysis. Moreover it is shown in [4] that MI-DP directly implies (ε, δ )-differential privacy.…”