Exploiting Explanations for Model Inversion Attacks

Abstract

The successful deployment of artificial intelligence (AI) in many domains from healthcare to hiring requires their responsible use, particularly in model explanations and privacy. Explainable artificial intelligence (XAI) provides more information to help users to understand model decisions, yet this additional knowledge exposes additional risks for privacy attacks. Hence, providing explanation harms privacy. We study this risk for image-based model inversion attacks and identified several attack architectures with increasing performance to reconstruct private image data from model explanations. We have developed several multi-modal transposed CNN architectures that achieve significantly higher inversion performance than using the target model prediction only. These XAI-aware inversion models were designed to exploit the spatial knowledge in image explanations. To understand which explanations have higher privacy risk, we analyzed how various explanation types and factors influence inversion performance. In spite of some models not providing explanations, we further demonstrate increased inversion performance even for nonexplainable target models by exploiting explanations of surrogate models through attention transfer. This method first inverts an explanation from the target prediction, then reconstructs the target image. These threats highlight the urgent and significant privacy risks of explanations and calls attention for new privacy preservation techniques that balance the dual-requirement for AI explainability and privacy.

Seven challenges for harmonizing explainability requirements

“…Research has shown that the output of feature importance methods like SHAP, through repeated queries, are prone to membership attacks that can reveal intimate details about the classification boundary [69]. Similar research has shown that counterfactual explanations are vulnerable to similar attacks [3], as are image-based explanations like saliency maps [89]. Such results reveal the risk that when providing explanations to external stakeholders, the recipients of such explanations can collude to reconstruct the inner workings of a model.…”

Towards Data-Free Model Stealing in a Hard Label Setting

“…In addition, recent works [29,33,36,43] have shown that an adversary could train a substitute model via model stealing and use it for crafting adversarial examples [12] in a black-box setting, which poses a serious threat when the model is deployed in security critical applications. A stolen model could also compromise the privacy of users by leaking confidential data through a membership inference attack [32] or model inversion [41,42]. Fig.…”