Against Membership Inference Attack: Pruning is All You Need

Wang, Yijue; Wang, Chenghong; Wang, Zigeng; Zhou, Shanglin; Liu, Huan; Bi, Jinbo; Ding, Caiwen; Rajasekaran, Sanguthevar

doi:10.48550/arxiv.2008.13578

Cited by 3 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is similar to confidence-based attacks as in [20,21]. [22][23][24][25][26][27] all study how various types of regularization can defend against MI attacks. Furthermore, MI is a special case of differential privacy (DP), and we refer the reader to [28] and [29] for a survey of DP results and its connections to deep learning, respectively, as well as to [30] for a precise connection between DP and MI.…”

Section: Related Workmentioning

confidence: 88%

See 1 more Smart Citation

Benign Overparameterization in Membership Inference with Early Stopping

Tan¹,

LeJeune²,

Mason³

et al. 2022

Preprint

View full text Add to dashboard Cite

Does a neural network's privacy have to be at odds with its accuracy? In this work, we study the effects the number of training epochs and parameters have on a neural network's vulnerability to membership inference (MI) attacks, which aim to extract potentially private information about the training data. We first demonstrate how the number of training epochs and parameters individually induce a privacy-utility trade-off: more of either improves generalization performance at the expense of lower privacy. However, remarkably, we also show that jointly tuning both can eliminate this privacy-utility trade-off. Specifically, with careful tuning of the number of training epochs, more overparameterization can increase model privacy for fixed generalization error. To better understand these phenomena theoretically, we develop a powerful new leave-one-out analysis tool to study the asymptotic behavior of linear classifiers and apply it to characterize the sample-specific loss threshold MI attack in high-dimensional logistic regression. For practitioners, we introduce a low-overhead procedure to estimate MI risk and tune the number of training epochs to guard against MI attacks.

show abstract

Section: Related Workmentioning

confidence: 88%

“…In this paper, we are interested in understanding both empirically and theoretically how overparameterization affects MI in classification. [25,34] show that pruning a network can improve MI robustness. [27] show empirically that MI tends to be easier on more challenging learning tasks.…”

Section: Related Workmentioning

confidence: 99%

Benign Overparameterization in Membership Inference with Early Stopping

Tan¹,

LeJeune²,

Mason³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Jia et al [20] studied to fuzzy the confidence score vector to evade the membership classification of the attack model. Wang et al [36] reduced model storage and computational operation to defend membership inference attacks. Yin et al [39] set definitions for utility and privacy of target classifier, and formulated the design goal of the defense method as an optimization problem and solved it.…”

Section: Protection Mechanism Against Membership Inference Attackmentioning

confidence: 99%

“…As a result, the prediction scores or the parameters' gradients in the target model are distinguishable for members and non-members of the training data. There are several defenses [31,27,29,33,26,32,36,39] have been proposed and most of them try to reducing overfitting by various regularization technique, such as L2 regularization [31], min-max game based adversarial regularization [26,39], and dropout [29]. In addition, Song et al [32] showed that early stopping outperformed other overfitting-prevention counter-measures against membership inference attacks to classifiers.…”

Section: Introductionmentioning

confidence: 99%

Privacy-preserving Generative Framework Against Membership Inference Attacks

Yang¹,

Ma²,

Miao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Artificial intelligence and machine learning have been integrated into all aspects of our lives and the privacy of personal data has attracted more and more attention. Since the generation of the model needs to extract the effective information of the training data, the model has the risk of leaking the privacy of the training data. Membership inference attacks can measure the model leakage of source data to a certain degree. In this paper, we design a privacy-preserving generative framework against membership inference attacks, through the information extraction and data generation capabilities of the generative model variational autoencoder (VAE) to generate synthetic data that meets the needs of differential privacy. Instead of adding noise to the model output or tampering with the training process of the target model, we directly process the original data. We first map the source data to the latent space through the VAE model to get the latent code, then perform noise process satisfying metric privacy on the latent code, and finally use the VAE model to reconstruct the synthetic data. Our experimental evaluation demonstrates that the machine learning model trained with newly generated synthetic data can effectively resist membership inference attacks and still maintain high utility.

show abstract

“…Existing empirical privacy defenses can be categorized by their method of protecting the training data (e.g., regularization [26,30], confidence-vector masking [20,49], knowledge distillation [44]). Alternatively, one can group defenses by whether they use only the private training data [44] or require access to reference data [20,26,30,40,48,49], defined as additional data from the same (or a similar) underlying distribution [30]. The two most prominent differentially private defenses can also be distinguished according to this distinction, where PATE [33] requires access to (unlabeled) reference data but DP-SGD [1] does not.…”

Section: Introductionmentioning

confidence: 99%

A Cautionary Tale: On the Role of Reference Data in Empirical Privacy Defenses

Kaplan,

Xu,

Marfoq

et al. 2024

PoPETs

View full text Add to dashboard Cite

Within the realm of privacy-preserving machine learning, empirical privacy defenses have been proposed as a solution to achieve satisfactory levels of training data privacy without a significant drop in model utility. Most existing defenses against membership inference attacks assume access to reference data, defined as an additional dataset coming from the same (or a similar) underlying distribution as training data. Despite the common use of reference data, previous works are notably reticent about defining and evaluating reference data privacy. As gains in model utility and/or training data privacy may come at the expense of reference data privacy, it is essential that all three aspects are duly considered. In this paper, we conduct the first comprehensive analysis of empirical privacy defenses. First, we examine the availability of reference data and its privacy treatment in previous works and demonstrate its necessity for fairly comparing defenses. Second, we propose a baseline defense that enables the utility-privacy tradeoff with respect to both training and reference data to be easily understood. Our method is formulated as an empirical risk minimization with a constraint on the generalization error, which, in practice, can be evaluated as a weighted empirical risk minimization (WERM) over the training and reference datasets. Although we conceived of WERM as a simple baseline, our experiments show that, surprisingly, it outperforms the most well-studied and current state-of-the-art empirical privacy defenses using reference data for nearly all relative privacy levels of reference and training data. Our investigation also reveals that these existing methods are unable to trade off reference data privacy for model utility and/or training data privacy, and thus fail to operate outside of the high reference data privacy case. Overall, our work highlights the need for a proper evaluation of the triad model utility / training data privacy / reference data privacy when comparing privacy defenses.

show abstract

Against Membership Inference Attack: Pruning is All You Need

Cited by 3 publications

References 26 publications

Benign Overparameterization in Membership Inference with Early Stopping

Benign Overparameterization in Membership Inference with Early Stopping

Privacy-preserving Generative Framework Against Membership Inference Attacks

A Cautionary Tale: On the Role of Reference Data in Empirical Privacy Defenses

Contact Info

Product

Resources

About