ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models

Abstract

Machine learning (ML) has become a core component of many real-world applications and training data is a key factor that drives current progress. This huge success has led Internet companies to deploy machine learning as a service (MLaaS). Recently, the first membership inference attack has shown that extraction of information on the training set is possible in such MLaaS settings, which has severe security and privacy implications.However, the early demonstrations of the feasibility of such attacks have many assumptions on the adversary, such as using multiple so-called shadow models, knowledge of the target model structure, and having a dataset from the same distribution as the target model's training data. We relax all these key assumptions, thereby showing that such attacks are very broadly applicable at low cost and thereby pose a more severe risk than previously thought. We present the most comprehensive study so far on this emerging and developing threat using eight diverse datasets which show the viability of the proposed attacks across domains.In addition, we propose the first effective defense mechanisms against such broader class of membership inference attacks that maintain a high level of utility of the ML model.

On the Resilience of Biometric Authentication Systems against Random Inputs

“…Membership inference attacks [24], [25] attempt to determine if a record obtained by an adversary was part of the original training data of the model. Whilst this attack does not compromise the security of the model, it breaches the privacy of the individual records.…”

“…These attacks create a shadow model [24] to mimic the behavior of the target model. Salem et al [25] construct a shadow model using only positive class samples and negative noise generated via uniformly random feature vectors. However it is hypothesized that these random samples belong to non-members, i.e., the negative class [25, §V.B].…”

“…3 We note that a key difference in the use of machine learning in biometric authentication as compared to its use in other areas (e.g., predicting likelihood of diseases through a healthcare dataset) is that the system should only output its decision: accept or reject [23], and not the detailed confidence values, i.e., confidence of the accept or reject decision. This makes our setting different from membership inference attacks where it is assumed that the model returns a prediction vector, where each element is the confidence (probability) that the associated class is the likely label of the input sample [24], [25]. In other words, less information is leaked in biometric authentication.…”

Effects of Differential Privacy and Data Skewness on Membership Inference Vulnerability

“…The attack technique proposed in [17], however, requires computational power and involves the training of multiple machine learning models. The techniques proposed in [18] are different still in that they require the attacker to develop effective threshold values. Figure 1 gives a workflow sketch of membership inference attack generation algorithm.…”

“…The membership inference attack against machine learning models was first presented in [17] where the authors proposed the shadow model attack. [14] characterized attack vulnerability with respect to different model types and datasets and introduced the vulnerability of loosely federated systems while the authors in [18] relax adversarial assumptions and generate a model and data independent attacker. Hayes et.…”