Differentially Private SQL with Bounded User Contribution

Wilson, Royce J; Zhang, Celia Yuxin; Lam, William H. K.; Desfontaines, Damien; Simmons-Marengo, Daniel; Gipson, Bryant

doi:10.48550/arxiv.1909.01917

Cited by 7 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Indeed, adjacency is often described by requiring that the Hamming distance between the n-tuples is at most 1.) Most implementations of DP are not explicit about these choices, but the wording in the documentation suggests the same conventions (e.g., see Section 1.3 in [49]).…”

Section: Underestimation Of Sensitivitymentioning

confidence: 99%

“…Google DP Library. Google's DP Library [49] was first released in September 2019 and remains under active development. It already supports many fundamental functions, such as count, sum, mean, variance, the Laplace mechanism, and the Gaussian mechanism, among others.…”

Section: An Overview Of Dp Librariesmentioning

confidence: 99%

“…(Although this attack still relies on having specific values in the dataset, a library that provides access to a "join" function like the one offered in [49] and a "resize" function like the one offered in [41] could enable a malicious data analyst to manipulate a dataset such that this attack could work in less specific settings.) Library-Mediated Attack 5.24 (SmartNoise; 32-bit Floats).…”

Section: Implementing Example Attacks 510 and 511mentioning

confidence: 99%

See 2 more Smart Citations

Widespread Underestimation of Sensitivity in Differentially Private Libraries and How to Fix It

Casacuberta¹,

Shoemate²,

Vadhan³

et al. 2022

Preprint

View full text Add to dashboard Cite

We identify a new class of vulnerabilities in implementations of differential privacy. Specifically, they arise when computing basic statistics such as sums, thanks to discrepancies between the implemented arithmetic using finite data types (namely, ints or floats) and idealized arithmetic over the reals or integers. These discrepancies result in the sensitivity of the implemented statistics -how much one individual's data can affect the result -to be much higher than the sensitivity we expect. Consequently, essentially all differential privacy libraries fail to introduce enough noise to hide individual-level information as required by differential privacy, and we show that this may be exploited in realistic attacks on differentially private query systems. In addition to presenting these vulnerabilities, we also provide a number of solutions, which modify or constrain the way in which the sum is implemented in order to recover the idealized or near-idealized bounds on sensitivity.

show abstract

Section: Underestimation Of Sensitivitymentioning

confidence: 99%

Section: An Overview Of Dp Librariesmentioning

confidence: 99%

Section: Implementing Example Attacks 510 and 511mentioning

confidence: 99%

See 1 more Smart Citation

Widespread Underestimation of Sensitivity in Differentially Private Libraries and How to Fix It

Casacuberta¹,

Shoemate²,

Vadhan³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Tools such as the Open Differential Privacy platform [21] aim to ease deployment of differential privacy for common scenarios. There has been recent research that enables global differential privacy on SQL analytic workloads [22] [23] [24]. Tools that are agnostic to compute targets and can scale to petabyte scale distributed computing environments enable the use of differentially private aggregates and censoring of rare dimension for computation of mutual information ranking.…”

Section: B Differential Privacymentioning

confidence: 99%

Distributed Differentially Private Mutual Information Ranking and Its Applications

Srivastava

Pouyanfar

Allen

et al. 2020

2020 IEEE 21st International Conference on Information Reuse and Integration for Data Science (IRI)

View full text Add to dashboard Cite

Computation of Mutual Information (MI) helps understand the amount of information shared between a pair of random variables. Automated feature selection techniques based on MI ranking are regularly used to extract information from sensitive datasets exceeding petabytes in size, over millions of features and classes. Series of one-vs-all MI computations can be cascaded to produce n-fold MI results, rapidly pinpointing informative relationships. This ability to quickly pinpoint the most informative relationships from datasets of billions of users creates privacy concerns. In this paper, we present Distributed Differentially Private Mutual Information (DDP-MI), a privacysafe fast batch MI, across various scenarios such as feature selection, segmentation, ranking, and query expansion. This distributed implementation is protected with global model differential privacy to provide strong assurances against a wide range of privacy attacks. We also show that our DDP-MI can substantially improve the efficiency of MI calculations compared to standard implementations on a large-scale public dataset.

show abstract

“…In machine learning, and specifically in the nascent field of federated learning (Konečnỳ et al, 2016) (see, e.g., (Kairouz et al, 2019) for a recent survey), private summation enables private Stochastic Gradient Descent (SGD), which in turn allows the private training of deep neural networks that are guaranteed not to overfit to any user-specific information. Moreover, summation is perhaps the most primitive functionality in database systems in general, and in private implementations in particular (see, e.g., (Kotsogiannis et al, 2019;Wilson et al, 2019;Suresh et al, 2017)).…”

Section: Introductionmentioning

confidence: 99%

Private Counting from Anonymous Messages: Near-Optimal Accuracy with Vanishing Communication Overhead

Ghazi,

Kumar,

Manurangsi

et al. 2021

Preprint

View full text Add to dashboard Cite

Differential privacy (DP) is a formal notion for quantifying the privacy loss of algorithms. Algorithms in the central model of DP achieve high accuracy but make the strongest trust assumptions whereas those in the local DP model make the weakest trust assumptions but incur substantial accuracy loss. The shuffled DP model (Bittau et al., 2017;Erlingsson et al., 2019;Cheu et al., 2019) has recently emerged as a feasible middle ground between the central and local models, providing stronger trust assumptions than the former while promising higher accuracies than the latter. In this paper, we obtain practical communication-efficient algorithms in the shuffled DP model for two basic aggregation primitives used in machine learning: 1) binary summation, and 2) histograms over a moderate number of buckets. Our algorithms achieve accuracy that is arbitrarily close to that of central DP algorithms with an expected communication per user essentially matching what is needed without any privacy constraints! We demonstrate the practicality of our algorithms by experimentally comparing their performance to several widely-used protocols such as Randomized Response (Warner, 1965) and RAPPOR (Erlingsson et al., 2014).

show abstract

Differentially Private SQL with Bounded User Contribution

Cited by 7 publications

References 18 publications

Widespread Underestimation of Sensitivity in Differentially Private Libraries and How to Fix It

Widespread Underestimation of Sensitivity in Differentially Private Libraries and How to Fix It

Distributed Differentially Private Mutual Information Ranking and Its Applications

Private Counting from Anonymous Messages: Near-Optimal Accuracy with Vanishing Communication Overhead

Contact Info

Product

Resources

About