Demystifying Membership Inference Attacks in Machine Learning as a Service

Abstract

Membership inference attacks seek to infer membership of individual training instances of a model to which an adversary has black-box access through a machine learning-as-a-service API. In providing an in-depth characterization of membership privacy risks against machine learning models, this paper presents a comprehensive study towards demystifying membership inference attacks from two complimentary perspectives. First, we provide a generalized formulation of the development of a black-box membership inference attack model. Second, we characterize the importance of model choice on model vulnerability through a systematic evaluation of a variety of machine learning models and model combinations using multiple datasets. Through formal analysis and empirical evidence from extensive experimentation, we characterize under what conditions a model may be vulnerable to such black-box membership inference attacks. We show that membership inference vulnerability is data-driven and corresponding attack models are largely transferable. Though different model types display different vulnerabilities to membership inference, so do different datasets. Our empirical results additionally show that (1) using the type of target model under attack within the attack model may not increase attack effectiveness and (2) collaborative learning exposes vulnerabilities to membership inference risks when the adversary is a participant. We also discuss countermeasure and mitigation strategies.

Deep Neural Network Ensembles Against Deception: Ensemble Diversity, Accuracy and Robustness

“…We also advocate multi-tier strategic ensemble learning [17,36]. We argue that such cross-layer hybrid approach can combine several ensemble strategies to further increase the diversity of ensemble members, providing more robust safeguards for the inputs to DNN models, the training of DNN models and the outputs of DNN models.…”

Effects of Differential Privacy and Data Skewness on Membership Inference Vulnerability

“…We note that all experiments evaluated the attack model against an equal number of instances in the target training dataset D as those not in D. The baseline membership inference accuracy is therefore 50%. We refer readers to [14] for more details on these datasets and experimental set up.…”

“…Figure 1 gives a workflow sketch of membership inference attack generation algorithm. We use the shadow model technique documented in [17] and [14] to describe the attack generation process of membership inference attacks, while noting that many of the processes may be applicable to other attack generation techniques.…”

“…1) Generating Shadow Data and Substitute Models: In the shadow model technique, an attacker must first generate or access a shadow dataset, a synthetic labeled dataset D to mirror the data in D. While [17] and [14] both outline potential approaches to generating such a synthetic dataset from scratch, we would like to note that in many cases, attackers may also have examples of their own which can be used as seeds for the shadow data generation process or to bootstrap their shadow dataset. Consider our example of the financial institution.…”

“…These results indicate that (1) the same strategy used for selecting the shadow model type may not be optimal for the attack model and (2) shadow models of different types other than the target model type may still lead to successful membership inference attacks. We refer readers to [14] for additional detail. The transferability study in Table III indicates an attacker does not always need to know the exact target model configuration to launch an effective membership inference attack as attack models can be transferable from one target model type to another.…”