Tunable Measures for Information Leakage and Applications to Privacy-Utility Tradeoffs

Abstract

We introduce a tunable measure for information leakage called maximal α-leakage. This measure quantifies the maximal gain of an adversary in inferring any (potentially random) function of a dataset from a release of the data. The inferential capability of the adversary is, in turn, quantified by a class of adversarial loss functions that we introduce as αloss, α ∈ [1, ∞) ∪ {∞}. The choice of α determines the specific adversarial action and ranges from refining a belief (about any function of the data) for α = 1 to guessing the most likely value for α = ∞ while refining the α th moment of the belief for α in between. Maximal α-leakage then quantifies the adversarial gain under α-loss over all possible functions of the data. In particular, for the extremal values of α = 1 and α = ∞, maximal αleakage simplifies to mutual information and maximal leakage, respectively. For α ∈ (1, ∞) this measure is shown to be the Arimoto channel capacity of order α. We show that maximal αleakage satisfies data processing inequalities and a sub-additivity property thereby allowing for a weak composition result. Building upon these properties, we use maximal α-leakage as the privacy measure and study the problem of data publishing with privacy guarantees, wherein the utility of the released data is ensured via a hard distortion constraint. Unlike average distortion, hard distortion provides a deterministic guarantee of fidelity. We show that under a hard distortion constraint, for α > 1 the optimal mechanism is independent of α, and therefore, the resulting optimal tradeoff is the same for all values of α > 1. Finally, the tunability of maximal α-leakage as a privacy measure is also illustrated for binary data with average Hamming distortion as the utility measure.

Bottleneck Problems: An Information and Estimation-Theoretic View

“…More in-depth analysis and properties of can be found in [ 103 ]. It is shown in [ 71 ] (Lemma 1) that for quantifies the minimum loss in recovering U given V where the loss is measured in terms of the so-called -loss. This loss function reduces to logarithmic loss ( 27 ) and for and , respectively.…”

“…As suggested in [ 55 , 62 ], one way to address this issue is to replace mutual information with other statistical measures. In the privacy literature, several measures with strong privacy guarantee have been proposed including Rényi maximal correlation [ 21 , 63 , 64 ], probability of correctly recovering [ 65 , 66 ], minimum mean-squared estimation error (MMSE) [ 67 , 68 ], -information [ 69 ] (a special case of f -information to be described in Section 3 ), Arimoto’s and Sibson’s mutual information [ 61 , 70 ]—to be discussed in Section 3 , maximal leakage [ 71 ], and local differential privacy [ 72 ]. All these measures ensure interpretable privacy guarantees.…”

Quantifying Membership Privacy via Information Leakage

“…For instance, [24] proposes to use the probability of correctly guessing the secret as a privacy metric. In [25] a class of tunable loss functions are introduced to capture a range of adversarial objectives, e.g., refining a belief or guessing the most likely value for the secret. Other methods include posing the privacy problem as a hypothesis test, e.g., in [26].…”

Using Rényi-divergence and Arimoto-Rényi Information to Quantify Membership Information Leakage

“…This measure was defined following an axiomatic framework for measuring information leakage requiring minimal assumptions, being interpretability, and satisfying data-processing, independence, and additivity properties. Maximal leakage was recently generalized to a family of leakage that can be finetuned to specific applications [19], [20]. In this pursuit, αleakage (when the adversary's target is known) and maximal α-leakage (when the adversary's target is not known) was introduced.…”

“…We show that these measures of information leakage are related to each other upon selecting α correctly. We compare these measures of information leakage with α-leakage from the privacy literature [19], [20]. This comparison allows us to establish quasi-convexity of α-information MIA information leakage.…”