Disparate Vulnerability to Membership Inference Attacks

Kulynych, Bogdan; Yaghini, Mohammad; Cherubin, Giovanni; Veale, Michael; Troncoso, Carmela

doi:10.2478/popets-2022-0023

Cited by 15 publications

(13 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When State value "CA" is updated to another common value "NY" (New York, comprising 8% of the dataset), the success rates by two-model attack is 6%, but when changed to a rare value "SD" (South Dakota, 0.2%), the attack success rates rise to 65%. This is inline with prior studies showing that membership inference attacks, for example, have higher success rate on records with rare values [12].…”

Section: Updated Attribute Inferencesupporting

confidence: 91%

Information Leakage from Data Updates in Machine Learning Models

Hui,

Farokhi,

Ohrimenko

2023

Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security

View full text Add to dashboard Cite

In this paper we consider the setting where machine learning models are retrained on updated datasets in order to incorporate the most up-to-date information or reflect distribution shifts. We investigate whether one can infer information about these updates in the training data (e.g., changes to attribute values of records).Here, the adversary has access to snapshots of the machine learning model before and after the change in the dataset occurs. Contrary to the existing literature, we assume that an attribute of a single or multiple training data points are changed rather than entire data records are removed or added. We propose attacks based on the difference in the prediction confidence of the original model and the updated model. We evaluate our attack methods on two public datasets along with multi-layer perceptron and logistic regression models. We validate that two snapshots of the model can result in higher information leakage in comparison to having access to only the updated model. Moreover, we observe that data records with rare values are more vulnerable to attacks, which points to the disparate vulnerability of privacy attacks in the update setting. When multiple records with the same original attribute value are updated to the same new value (i.e., repeated changes), the attacker is more likely to correctly guess the updated values since repeated changes leave a larger footprint on the trained model. These observations point to vulnerability of machine learning models to attribute inference attacks in the update setting. CCS CONCEPTS• Security and privacy; • Computing methodologies → Machine learning;

show abstract

Section: Updated Attribute Inferencesupporting

confidence: 91%

Information Leakage from Data Updates in Machine Learning Models

Hui,

Farokhi,

Ohrimenko

2023

Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security

View full text Add to dashboard Cite

show abstract

“…We are not able to answer these questions yet, although our experiments provide several intriguing observations and suggest possibilities to explore. It is not surprising that so little is understood about distribution inference-the research community has put extensive effort into studying membership inference attacks, and we are just beginning to be able to understand how and why membership inference risk varies [32]. There could also be trade-offs between robustness, fairness, interpretability, and vulnerability to distribution inference attacks.…”

Section: Discussionmentioning

confidence: 99%

Formalizing and Estimating Distribution Inference Risks

Suri

Evans

2022

PoPETs

View full text Add to dashboard Cite

Distribution inference, sometimes called property inference, infers statistical properties about a training set from access to a model trained on that data. Distribution inference attacks can pose serious risks when models are trained on private data, but are difficult to distinguish from the intrinsic purpose of statistical machine learning—namely, to produce models that capture statistical properties about a distribution. Motivated by Yeom et al.’s membership inference framework, we propose a formal definition of distribution inference attacks general enough to describe a broad class of attacks distinguishing between possible training distributions. We show how our definition captures previous ratio-based inference attacks as well as new kinds of attack including revealing the average node degree or clustering coefficient of training graphs. To understand distribution inference risks, we introduce a metric that quantifies observed leakage by relating it to the leakage that would occur if samples from the training distribution were provided directly to the adversary. We report on a series of experiments across a range of different distributions using both novel black-box attacks and improved versions of the state-of-the-art white-box attacks. Our results show that inexpensive attacks are often as effective as expensive meta-classifier attacks, and that there are surprising asymmetries in the effectiveness of attacks.

show abstract

“…Empirically, this is evaluated through membership inference attacks, where an attacker uses the model to determine whether a given data point was in the training set (Shokri et al, 2017). While Kulynych et al (2022) observed that DP reduces disparate vulnerability to such attacks, it has also been shown that DP can exacerbate unfairness (Bagdasaryan et al, 2019;Pujol et al, 2020). Conversely, Chang and Shokri (2020) showed that enforcing a fair model leads to more privacy leakage for the unprivileged group.…”

Section: Related Workmentioning

confidence: 99%

Fair NLP Models with Differentially Private Text Encoders

Maheshwari¹,

Pelletier²,

Keller³

et al. 2022

Preprint

View full text Add to dashboard Cite

Encoded text representations often capture sensitive attributes about individuals (e.g., race or gender), which raise privacy concerns and can make downstream models unfair to certain groups. In this work, we propose FEDERATE, an approach that combines ideas from differential privacy and adversarial training to learn private text representations which also induces fairer models. We empirically evaluate the trade-off between the privacy of the representations and the fairness and accuracy of the downstream model on four NLP datasets. Our results show that FEDERATE consistently improves upon previous methods, and thus suggest that privacy and fairness can positively reinforce each other.

show abstract

Disparate Vulnerability to Membership Inference Attacks

Cited by 15 publications

References 19 publications

Information Leakage from Data Updates in Machine Learning Models

Information Leakage from Data Updates in Machine Learning Models

Formalizing and Estimating Distribution Inference Risks

Fair NLP Models with Differentially Private Text Encoders

Contact Info

Product

Resources

About