Wide Network Learning with Differential Privacy

Zhang, Huanyu; Mironov, Ilya; Hejazinia, Meisam

doi:10.48550/arxiv.2103.01294

Cited by 8 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, as discussed in Section 3, hybrid clipping with 𝑙 ∞ -norm constraint is only determined by a few stable aggregate statistics, such as the principal components and the average of the power in each of them, which capture the populational statistics of underlying processed output distribution. This is a much more smooth operation compared to many other existing clipping methods, such as sparsification [34,51,53], where only significant coordinates are preserved or participate in the processing while the remaining are either frozen or removed. Though these artificial dimension-reduction techniques can also decrease the noise scale, the advantage can be easily offset by the large clipping bias produced and may not outperform simple 𝑙 2 -norm clipping, especially in deep learning [11].…”

Section: A Additional Discussionmentioning

confidence: 99%

“…where 𝑒 (𝑡 ) is some Gaussian noise. Running for 𝑇 iterations with a total privacy budget (𝜖, 𝛿), one may select 𝑒 (𝑡 ) ∼ N (0, 𝜎 Another critical motivation behind these experiments is to evaluate the performance of classic dimension-reduction clipping methods, such as sparsification [34], [51], [53] (preserving only significant coordinates) or low-rank embedding [50] (projection to a subspace). From a theoretical perspective, these strategies can artificially alleviate the curse of dimensionality, as the scale of noise is now determined by the Hamming weight after sparsification or the rank of embedding.…”

Section: Preliminariesmentioning

confidence: 99%

“…However, there is a large gap between theory and practice. Those good properties do not hold for many complicated processing tasks, and artificial approximation such as sparsification may cause large bias [34,51,53]. 𝑙 2 -norm clipping [11] and its variants, such as layer clipping [35] or subspace embedding clipping [50], where the objective is split into several segments and each is 𝑙 2 -norm clipped with possibly different parameters, are still the most popular options, especially for deep learning.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Geometry of Sensitivity: Twice Sampling and Hybrid Clipping in Differential Privacy with Optimal Gaussian Noise and Application to Deep Learning

Xiao,

Wan,

Devadas

2023

Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

We study the fundamental problem of the construction of optimal randomization in Differential Privacy (DP). Depending on the clipping strategy or additional properties of the processing function, the corresponding sensitivity set theoretically determines the necessary randomization to produce the required security parameters. Towards the optimal utility-privacy tradeoff, finding the minimal perturbation for properly-selected sensitivity sets stands as a central problem in DP research. In practice, 𝑙 2 /𝑙 1 -norm clippings with Gaussian/Laplace noise mechanisms are among the most common setups. However, they also suffer from the curse of dimensionality. For more generic clipping strategies, the understanding of the optimal noise for a high-dimensional sensitivity set remains limited. This raises challenges in mitigating the worst-case dimension dependence in privacy-preserving randomization, especially for deep learning applications.In this paper, we revisit the geometry of high-dimensional sensitivity sets and present a series of results to characterize the nonasymptotically optimal Gaussian noise for Rényi DP (RDP). Our results are both negative and positive: on one hand, we show the curse of dimensionality is tight for a broad class of sensitivity sets satisfying certain symmetry properties; but if, fortunately, the representation of the sensitivity set is asymmetric on some group of orthogonal bases, we show the optimal noise bounds need not be explicitly dependent on either dimension or rank. We also revisit sampling in the high-dimensional scenario, which is the key for both privacy amplification and computation efficiency in largescale data processing. We propose a novel method, termed twice sampling, which implements both sample-wise and coordinate-wise sampling, to enable Gaussian noises to fit the sensitivity geometry more closely. With closed-form RDP analysis, we prove twice sampling produces asymptotic improvement of the privacy amplification given an additional 𝑙 ∞ -norm restriction, especially for small sampling rate. We also provide concrete applications of our results on practical tasks. Through tighter privacy analysis combined with twice sampling, we efficiently train ResNet22 in low sampling rate on CIFAR10, and achieve 69.7% and 81.6% test accuracy with (𝜖 = 2, 𝛿 = 10 −5 ) and (𝜖 = 8, 𝛿 = 10 −5 ) DP guarantee, respectively. CCS CONCEPTS• Security and privacy → Privacy-preserving protocols;

show abstract

Section: A Additional Discussionmentioning

confidence: 99%

Section: Preliminariesmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Geometry of Sensitivity: Twice Sampling and Hybrid Clipping in Differential Privacy with Optimal Gaussian Noise and Application to Deep Learning

Xiao,

Wan,

Devadas

2023

Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

show abstract

“…However the authors point out that high degree of noise motivated by ensuring high-level privacy directly impacts the relative ranking of models' performances. Most recent works are extending differential privacy methods to complex deep recommender systems such as wide and deep architectures [30] and collaborative bandits learning [26].…”

Section: Privacymentioning

confidence: 99%

Synthetic Data and Simulators for Recommendation Systems: Current State and Future Directions

Lesnikowski¹,

Moreira²,

Rabhi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Synthetic data and simulators have the potential to markedly improve the performance and robustness of recommendation systems.These approaches have already had a beneficial impact in other machine-learning driven fields. We identify and discuss a key trade-off between data fidelity and privacy in the past work on synthetic data and simulators for recommendation systems. For the important use case of predicting algorithm rankings on real data from synthetic data, we provide motivation and current successes versus limitations. Finally we outline a number of exciting future directions for recommendation systems that we believe deserve further attention and work, including mixing real and synthetic data, feedback in dataset generation, robust simulations, and privacy-preserving methods.

show abstract

“…Application. A common use case for SVT is the differentially private release of only those entries of a vector with large magnitude, instead of the entire vector [27,39]. This can be desirable for multiple reasons: to be able to release the entries with less noise, since the privacy budget needs to be divided between fewer entries; to release only those values of a histogram that are large enough in magnitude so that they will not be dominated by the added noise; or to reduce communication costs in a distributed setting.…”

mentioning

confidence: 99%

Privacy accounting εconomics: Improving differential privacy composition via a posteriori bounds

Hartmann

Bindschaedler

Bentkamp

et al. 2022

PoPETs

View full text Add to dashboard Cite

Differential privacy (DP) is a widely used notion for reasoning about privacy when publishing aggregate data. In this paper, we observe that certain DP mechanisms are amenable to a posteriori privacy analysis that exploits the fact that some outputs leak less information about the input database than others. To exploit this phenomenon, we introduce output differential privacy (ODP) and a new composition experiment, and leverage these new constructs to obtain significant privacy budget savings and improved privacy–utility tradeoffs under composition. All of this comes at no cost in terms of privacy; we do not weaken the privacy guarantee. To demonstrate the applicability of our a posteriori privacy analysis techniques, we analyze two well-known mechanisms: the Sparse Vector Technique and the Propose-Test-Release framework. We then show how our techniques can be used to save privacy budget in more general contexts: when a differentially private iterative mechanism terminates before its maximal number of iterations is reached, and when the output of a DP mechanism provides unsatisfactory utility. Examples of the former include iterative optimization algorithms, whereas examples of the latter include training a machine learning model with a large generalization error. Our techniques can be applied beyond the current paper to refine the analysis of existing DP mechanisms or guide the design of future mechanisms.

show abstract

Wide Network Learning with Differential Privacy

Cited by 8 publications

References 30 publications

Geometry of Sensitivity: Twice Sampling and Hybrid Clipping in Differential Privacy with Optimal Gaussian Noise and Application to Deep Learning

Geometry of Sensitivity: Twice Sampling and Hybrid Clipping in Differential Privacy with Optimal Gaussian Noise and Application to Deep Learning

Synthetic Data and Simulators for Recommendation Systems: Current State and Future Directions

Privacy accounting εconomics: Improving differential privacy composition via a posteriori bounds

Contact Info

Product

Resources

About