GAMIN: An Adversarial Approach to Black-Box Model Inversion

Aïvodji, Ulrich; Gambs, Sébastien; Ther, Timon

doi:10.48550/arxiv.1909.11835

Cited by 8 publications

(28 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While for the black-box scenario, there are no impressive works before the birth of GAN. In [3], a model inversion attack framework was built under the black-box setting. Given a target model 𝑓 and a label 𝑦 𝑡 , an attacker aims at characterizing data 𝑥 𝑡 belonging to 𝑦 𝑡 .…”

Section: Attacks On Preimagementioning

confidence: 99%

Generative Adversarial Networks: A Survey Towards Private and Secure Applications

Cai,

Xiong,

et al. 2021

Preprint

View full text Add to dashboard Cite

Generative Adversarial Networks (GAN) have promoted a variety of applications in computer vision, natural language processing, etc. due to its generative model's compelling ability to generate realistic examples plausibly drawn from an existing distribution of samples. GAN not only provides impressive performance on data generation-based tasks but also stimulates fertilization for privacy and security oriented research because of its game theoretic optimization strategy. Unfortunately, there are no comprehensive surveys on GAN in privacy and security, which motivates this survey paper to summarize those state-of-the-art works systematically. The existing works are classified into proper categories based on privacy and security functions, and this survey paper conducts a comprehensive analysis of their advantages and drawbacks. Considering that GAN in privacy and security is still at a very initial stage and has imposed unique challenges that are yet to be well addressed, this paper also sheds light on some potential privacy and security applications with GAN and elaborates on some future research directions. CCS Concepts: • General and reference → Surveys and overviews; • Security and privacy; • Computing methodologies → Machine learning;

show abstract

Section: Attacks On Preimagementioning

confidence: 99%

Generative Adversarial Networks: A Survey Towards Private and Secure Applications

Cai,

Xiong,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This scenario has been left as an open problem. Aïvodji et al [21] have introduced a generative adversarial model inversion under the black-box scenario, and they have achieved considerable results even against a deep model. Although their approach does not have any information regarding the model system, it is based on try and error based on assumptions of target model structure as well for the data structure.…”

Section: Related Workmentioning

confidence: 99%

“…Table 1 gives a brief review to the above mentieond related works. [15] MIA in privacy violation in pharmacogenetics by Fredrikson et al [18] Initial introduciton of MIA by Fredrikson et al [19] Stealing machine learning models via prediction apis by Tramèr et al [20] Discussion on black-box scenario [21] Introduced a generative adversarial model inversion under the black-box scenario by Aïvodji et al [23,24] Model inversion attacks for prediction systems: Without knowledge of non-sensitive attributes by Hidano et al [25] Using generative models in MIA on deep neural networks by Zhang et al [26] Generating and implementing optimum seed image by pre-trained GAN for initialization of MIA process on DL based recognition system by Khosravy et al…”

Section: Related Workmentioning

confidence: 99%

Model Inversion Attack: Analysis under Gray-box Scenario on Deep Learning based Face Recognition System

2021

KSII TIIS

View full text Add to dashboard Cite

In a wide range of ML applications, the training data contains privacy-sensitive information that should be kept secure. Training the ML systems by privacy-sensitive data makes the ML model inherent to the data. As the structure of the model has been fine-tuned by training data, the model can be abused for accessing the data by the estimation in a reverse process called model inversion attack (MIA). Although, MIA has been applied to shallow neural network models of recognizers in literature and its threat in privacy violation has been approved, in the case of a deep learning (DL) model, its efficiency was under question. It was due to the complexity of a DL model structure, big number of DL model parameters, the huge size of training data, big number of registered users to a DL model and thereof big number of class labels. This research work first analyses the possibility of MIA on a deep learning model of a recognition system, namely a face recognizer. Second, despite the conventional MIA under the white box scenario of having partial access to the users' non-sensitive information in addition to the model structure, the MIA is implemented on a deep face recognition system by just having the model structure and parameters but not any user information. In this aspect, it is under a semi-white box scenario or in other words a gray-box scenario. The experimental results in targeting five registered users of a CNN-based face recognition system approve the possibility of regeneration of users' face images even for a deep model by MIA under a gray box scenario. Although, for some images the evaluation recognition score is low and the generated images are not easily recognizable, but for some other images the score is high and facial features of the targeted identities are observable. The objective and subjective evaluations demonstrate that privacy cyber-attack by MIA on a deep recognition system not only is feasible but also is a serious threat with increasing alert state in the future as there is considerable potential for integration more advanced ML techniques to MIA.

show abstract

“…Unfortunately, since ML models tend to memorize information about training data, even when stored and processed securely, privacy information can still be exposed through the access to the models [21]. Indeed, the prior study of privacy attacks has demonstrated the possibility of exposing training data at different granularities, ranging from "coarse-grained" information such as determining whether a certain point participate in training [11,15,17,22] or whether a training dataset satisfies certain properties [10,16], to more "fine-grained" information such as reconstructing the raw data [2,4,8,25].…”

Section: Introductionmentioning

confidence: 99%

“…The synthesis is implemented as a gradient ascent algorithm. By contrast, existing blackbox attacks [2,20] are based on training an attack network that predicts the sensitive feature from the input confidence scores. Despite the exclusive focus on these two threat models, in practice, ML models are often packed into a blackbox that only produces hard-labels when being queried.…”

Section: Introductionmentioning

confidence: 99%

Label-Only Model Inversion Attacks via Boundary Repulsion

Kahla¹,

Chen²,

Anh³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent studies show that the state-of-the-art deep neural networks are vulnerable to model inversion attacks, in which access to a model is abused to reconstruct private training data of any given target class. Existing attacks rely on having access to either the complete target model (whitebox) or the model's soft-labels (blackbox). However, no prior work has been done in the harder but more practical scenario, in which the attacker only has access to the model's predicted label, without a confidence measure. In this paper, we introduce an algorithm, Boundary-Repelling Model Inversion (BREP-MI), to invert private training data using only the target model's predicted labels. The key idea of our algorithm is to evaluate the model's predicted labels over a sphere and then estimate the direction to reach the target class's centroid. Using the example of face recognition, we show that the images reconstructed by BREP-MI successfully reproduce the semantics of the private training data for various datasets and target model architectures. We compare BREP-MI with the state-of-the-art whitebox and blackbox model inversion attacks and the results show that despite assuming less knowledge about the target model, BREP-MI outperforms the blackbox attack and achieves comparable results to the whitebox attack.

show abstract

GAMIN: An Adversarial Approach to Black-Box Model Inversion

Cited by 8 publications

References 35 publications

Generative Adversarial Networks: A Survey Towards Private and Secure Applications

Generative Adversarial Networks: A Survey Towards Private and Secure Applications

Model Inversion Attack: Analysis under Gray-box Scenario on Deep Learning based Face Recognition System

Label-Only Model Inversion Attacks via Boundary Repulsion

Contact Info

Product

Resources

About