Model extraction from counterfactual explanations

Aïvodji, Ulrich; Bolot, Alexandre; Gambs, Sébastien

doi:10.48550/arxiv.2009.01884

Cited by 8 publications

(26 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We explain this attack in detail in Section 3. [8,2]. Milli et al proposed an attack that used a loss regarding the distance between a gradient-based explanation of a victim model and that of a clone model and experimentally showed that the loss improved the efficiency of the attack [8].…”

Section: Related Workmentioning

confidence: 99%

“…Milli et al proposed an attack that used a loss regarding the distance between a gradient-based explanation of a victim model and that of a clone model and experimentally showed that the loss improved the efficiency of the attack [8]. Ulrich et al proposed an attack that used counterfactual explanations of the targeted model for training the clone model efficiently [2].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AI

Miura,

Hasegawa,

Shibahara

2021

Preprint

View full text Add to dashboard Cite

The advance of explainable artificial intelligence, which provides reasons for its predictions, is expected to accelerate the use of deep neural networks in the real world like Machine Learning as a Service (MLaaS) that returns predictions on queried data with the trained model. Deep neural networks deployed in MLaaS face the threat of model extraction attacks. A model extraction attack is an attack to violate intellectual property and privacy in which an adversary steals trained models in a cloud using only their predictions. In particular, a data-free model extraction attack has been proposed recently and is more critical. In this attack, an adversary uses a generative model instead of preparing input data. The feasibility of this attack, however, needs to be studied since it requires more queries than that with surrogate datasets. In this paper, we propose MEGEX, a data-free model extraction attack against a gradient-based explainable AI. In this method, an adversary uses the explanations to train the generative model and reduces the number of queries to steal the model. Our experiments show that our proposed method reconstructs high-accuracy models -0.97× and 0.98× the victim model accuracy on SVHN and CIFAR-10 datasets given 2M and 20M queries, respectively. This implies that there is a trade-off between the interpretability of models and the difficulty of stealing them.1 https://cloud.google.com/explainable-ai Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AI

Miura,

Hasegawa,

Shibahara

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Membership inference attacks can be improved by concatenating gradient explanations with model predictions [38]. Model extraction attacks can be improved by regularizing the reconstructed model with gradient explanations [29], and training a surrogate model from counterfactual explanation examples [2]. However, exploiting explanations for model inversion attacks remains unexplored; this conceals the privacy risk on user data at prediction time, i.e., of active users.…”

Section: Related Workmentioning

confidence: 99%

“…where x a and x b are two images being compared, µ * and σ * represents the pixel value mean and standard deviation, respectively, C µ = (K µ L) 2 and C σ = (K σ L) 2 are constants to control instability, L is the dynamic range of the pixel values (255 for 8-bit grayscale images), and K µ = 0.01 and K σ = 0.03 are chosen to be small. To compare images at different levels of granularity, we compare Gaussian kernels for both images at specified standard deviations, σ (smaller σ for more precise comparison), and compute their mean.…”

Section: Evaluation Metricsmentioning

confidence: 99%

“…Recent work has begun to study the privacy threats of model explanations, but their attack goals differ and their use of explanations remains under-studied. Instead of attacking training data [39] or model confidentiality [2,29], we focus on attacking the privacy of test instances. This will compromise the trust of active users of the model.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Exploiting Explanations for Model Inversion Attacks

Zhao¹,

Zhang²,

Xiao³

et al. 2021

Preprint

View full text Add to dashboard Cite

The successful deployment of artificial intelligence (AI) in many domains from healthcare to hiring requires their responsible use, particularly in model explanations and privacy. Explainable artificial intelligence (XAI) provides more information to help users to understand model decisions, yet this additional knowledge exposes additional risks for privacy attacks. Hence, providing explanation harms privacy. We study this risk for image-based model inversion attacks and identified several attack architectures with increasing performance to reconstruct private image data from model explanations. We have developed several multi-modal transposed CNN architectures that achieve significantly higher inversion performance than using the target model prediction only. These XAI-aware inversion models were designed to exploit the spatial knowledge in image explanations. To understand which explanations have higher privacy risk, we analyzed how various explanation types and factors influence inversion performance. In spite of some models not providing explanations, we further demonstrate increased inversion performance even for nonexplainable target models by exploiting explanations of surrogate models through attention transfer. This method first inverts an explanation from the target prediction, then reconstructs the target image. These threats highlight the urgent and significant privacy risks of explanations and calls attention for new privacy preservation techniques that balance the dual-requirement for AI explainability and privacy.

show abstract

Toward Explainable Affective Computing: A Review

Cortiñas-Lorenzo,

Lacey

2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Affective computing has an unprecedented potential to change the way humans interact with technology. While the last decades have witnessed vast progress in the field, multimodal affective computing systems are generally black box by design. As affective systems start to be deployed in real-world scenarios, such as education or healthcare, a shift of focus toward improved transparency and interpretability is needed. In this context, how do we explain the output of affective computing models? and how to do so without limiting predictive performance? In this article, we review affective computing work from an explainable AI (XAI) perspective, collecting and synthesizing relevant papers into three major XAI approaches: premodel (applied before training), in-model (applied during training), and postmodel (applied after training). We present and discuss the most fundamental challenges in the field, namely, how to relate explanations back to multimodal and time-dependent data, how to integrate context and inductive biases into explanations using mechanisms such as attention, generative modeling, or graph-based methods, and how to capture intramodal and cross-modal interactions in post hoc explanations. While explainable affective computing is still nascent, existing methods are promising, contributing not only toward improved transparency but, in many cases, surpassing state-of-the-art results. Based on these findings, we explore directions for future research and discuss the importance of data-driven XAI and explanation goals, and explainee needs definition, as well as causability or the extent to which a given method leads to human understanding.

show abstract

Model extraction from counterfactual explanations

Cited by 8 publications

References 65 publications

MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AI

MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AI

Exploiting Explanations for Model Inversion Attacks

Toward Explainable Affective Computing: A Review

Contact Info

Product

Resources

About