Interpretability is a Kind of Safety: An Interpreter-based Ensemble for Adversary Defense

Wang, Jingyuan; Yan-ru, WU; Li, Mingxuan; Lin, Xin; Wu, Junjie; Li, Chao

doi:10.1145/3394486.3403044

Cited by 10 publications

(1 citation statement)

References 30 publications

(68 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we further explore the premise of using explanations, but to detect adversarial examples and based on a fundamentally different approach. Our work was done concurrently with a similar approach presented by Wang et al [48]. In contrast, we have the following differentiating aspects.…”

Section: Existing Work On Adversarial Detectionmentioning

confidence: 99%

ExAD: An Ensemble Approach for Explanation-based Adversarial Detection

Vardhan¹,

Liu²,

Chinprutthiwong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent research has shown Deep Neural Networks (DNNs) to be vulnerable to adversarial examples that induce desired misclassifications in the models. Such risks impede the application of machine learning in security-sensitive domains. Several defense methods have been proposed against adversarial attacks to detect adversarial examples at test time or to make machine learning models more robust. However, while existing methods are quite effective under blackbox threat model, where the attacker is not aware of the defense, they are relatively ineffective under whitebox threat model, where the attacker has full knowledge of the defense.In this paper, we propose ExAD, a framework to detect adversarial examples using an ensemble of explanation techniques. Each explanation technique in ExAD produces an explanation map identifying the relevance of input variables for the model's classification. For every class in a dataset, the system includes a detector network, corresponding to each explanation technique, which is trained to distinguish between normal and abnormal explanation maps. At test time, if the explanation map of an input is detected as abnormal by any detector model of the classified class, then we consider the input to be an adversarial example. We evaluate our approach using six state-of-the-art adversarial attacks on three image datasets. Our extensive evaluation shows that our mechanism can effectively detect these attacks under blackbox threat model with limited false-positives. Furthermore, we find that our approach achieves promising results in limiting the success rate of whitebox attacks.

show abstract

Section: Existing Work On Adversarial Detectionmentioning

confidence: 99%

ExAD: An Ensemble Approach for Explanation-based Adversarial Detection

Vardhan¹,

Liu²,

Chinprutthiwong³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Adversarial Attacks in Explainable Machine Learning: A Survey of Threats Against Models and Humans

Vadillo,

Santana,

Lozano

2024

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Reliable deployment of machine learning models such as neural networks continues to be challenging due to several limitations. Some of the main shortcomings are the lack of interpretability and the lack of robustness against adversarial examples or out‐of‐distribution inputs. In this paper, we comprehensively review the possibilities and limits of adversarial attacks for explainable machine learning models. First, we extend the notion of adversarial examples to fit in explainable machine learning scenarios where a human assesses not only the input and the output classification, but also the explanation of the model's decision. Next, we propose a comprehensive framework to study whether (and how) adversarial examples can be generated for explainable models under human assessment. Based on this framework, we provide a structured review of the diverse attack paradigms existing in this domain, identify current gaps and future research directions, and illustrate the main attack paradigms discussed. Furthermore, our framework considers a wide range of relevant yet often ignored factors such as the type of problem, the user expertise or the objective of the explanations, in order to identify the attack strategies that should be adopted in each scenario to successfully deceive the model (and the human). The intention of these contributions is to serve as a basis for a more rigorous and realistic study of adversarial examples in the field of explainable machine learning.

show abstract

Mitigating adversarial perturbations via weakly supervised object location and regions recombination

Guo

Zhang³

et al. 2022

Machine Vision and Applications

View full text Add to dashboard Cite

Interpretability is a Kind of Safety: An Interpreter-based Ensemble for Adversary Defense

Cited by 10 publications

References 30 publications

ExAD: An Ensemble Approach for Explanation-based Adversarial Detection

ExAD: An Ensemble Approach for Explanation-based Adversarial Detection

Adversarial Attacks in Explainable Machine Learning: A Survey of Threats Against Models and Humans

Mitigating adversarial perturbations via weakly supervised object location and regions recombination

Contact Info

Product

Resources

About