Deep Learning with Gaussian Differential Privacy

Abstract

Deep learning models are often trained on datasets that contain sensitive information such as individuals' shopping transactions, personal contacts, and medical records. An increasingly important line of work therefore has sought to train neural networks subject to privacy constraints that are specified by differential privacy or its divergence-based relaxations. These privacy definitions, however, have weaknesses in handling certain important primitives (composition and subsampling), thereby giving loose or complicated privacy analyses of training neural networks. In this paper, we consider a recently proposed privacy definition termed f -differential privacy [17] for a refined privacy analysis of training neural networks. Leveraging the appealing properties of f -differential privacy in handling composition and subsampling, this paper derives analytically tractable expressions for the privacy guarantees of both stochastic gradient descent and Adam used in training deep neural networks, without the need of developing sophisticated techniques as [3] did. Our results demonstrate that the f -differential privacy framework allows for a new privacy analysis that improves on the prior analysis [3], which in turn suggests tuning certain parameters of neural networks for a better prediction accuracy without violating the privacy budget. These theoretically derived improvements are confirmed by our experiments in a range of tasks in image classification, text classification, and recommender systems.

On the Convergence and Calibration of Deep Learning with Differential Privacy

“…We now introduce the DP optimizers [2,1] to train the DP neural networks. One popular optimizer is the DP-SGD [55,16,3,9] in Algorithm 1 and more optimizers such as DP-Adam can be found in Appendix F. In contrast to the standard SGD, the DP-SGD has two unique steps: the per-sample clipping (to guarantee the sensitivity of per-sample gradients) and the random noise addition (to guarantee the privacy of models), both discussed in details via the Gaussian mechanism in Lemma 5.2.…”

“…For the same differentially private mechanism, different privacy accountants (e.g., Moments accountant [3,13], Gaussian differential privacy (GDP) [24,9], Fourier accountant [38], each based on a different composition theory) accumulate the privacy risk (σ, n, p, δ, T ) differently over T iterations. The next result shows that DP optimizers with global clipping is as private as those with local clipping, independent of the choice of the privacy accountant.…”

“…These optimizers are supported in Opacus and Tensorflow Privacy libraries. The original form of DP-Adam can be found in [9].…”

“…To achieve this gold standard of privacy guarantee, since the seminal work [3], DP optimizers are applied to train the neural networks while preserving the accuracy of prediction. To name a few, researchers have proposed DP-SGD [3,8], DP-Adam [9], DP-SGLD [56,40], DP-FTRL [36], and DP-FedAvg [42].…”

Data-Free Evaluation of User Contributions in Federated Learning

“…However, there may exist strategic users who can manipulate this process to achieve their own interests, e.g., by uploading a fake model update xi = x i where x i is the true model update (omitting subscript t for notation simplicity). We account for two main types of strategic user behaviors in FL: 1) free riding [6], which generates random model parameters without actually training to save training costs, such as computing power and storage; and 2) overly privacy-preserving, which adds excessive noises to the model parameters for privacy protection [11]- [13]).…”