Resilience of Bayesian Layer-Wise Explanations under Adversarial Attacks

Carbone, Ginevra; Sanguinetti, Guido; Bortolussi, Luca

doi:10.1109/ijcnn55064.2022.9892788

Cited by 5 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Aside from the problem of gradient masking, the distinction between category I and II attacks is also useful because certain types of neural networks can be shown to be immune to gradient-based attacks. Carbone et al [52], for example, obtained a very interesting result proving that Bayesian neural networks are immune to gradient-based adversarial attacks in the infinite data limit, because the gradient of the loss with respect to any sample from the data distribution is zero. This result also seems to hold approximately for Bayesian neural networks trained on finite data.…”

Section: Randomized Gradient-free Attackmentioning

confidence: 99%

An Introduction to Adversarially Robust Deep Learning

Peck,

Goossens,

Saeys

2024

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

The widespread success of deep learning in solving machine learning problems has fueled its adoption in many fields, from speech recognition to drug discovery and medical imaging. However, deep learning systems are extremely fragile: imperceptibly small modifications to their input data can cause the models to produce erroneous output. It is very easy to generate such adversarial perturbations even for state-of-the-art models, yet immunization against them has proven exceptionally challenging. Despite over a decade of research on this problem, our solutions are still far from satisfactory and many open problems remain. In this work, we survey some of the most important contributions in the field of adversarial robustness. We pay particular attention to the reasons why past attempts at improving robustness have been insufficient, and we identify several promising areas for future research.

show abstract

Section: Randomized Gradient-free Attackmentioning

confidence: 99%

An Introduction to Adversarially Robust Deep Learning

Peck,

Goossens,

Saeys

2024

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Classification performance has not significantly improved despite the widespread use of de-noiser models to reduce adversarial noise [159]. For example, Carbone et al demonstrate the stability of saliency-based explanations of Neural Network predictions under adversarial attacks in a classification task [161]. The authors implement a gradient-based XAI method using Bayesian Neural Networks, which is considerably more stable under adversarial perturbations of the inputs and even under direct attacks on the explanations.…”

Section: Gradient-based Adversarial Explanationmentioning

confidence: 99%

An Empirical Survey on Explainable AI Technologies: Recent Trends, Use-Cases, and Categories from Technical and Application Perspectives

et al. 2023

View full text Add to dashboard Cite

In a wide range of industries and academic fields, artificial intelligence is becoming increasingly prevalent. AI models are taking on more crucial decision-making tasks as they grow in popularity and performance. Although AI models, particularly machine learning models, are successful in research, they have numerous limitations and drawbacks in practice. Furthermore, due to the lack of transparency behind their behavior, users need more understanding of how these models make specific decisions, especially in complex state-of-the-art machine learning algorithms. Complex machine learning systems utilize less transparent algorithms, thereby exacerbating the problem. This survey analyzes the significance and evolution of explainable AI (XAI) research across various domains and applications. Throughout this study, a rich repository of explainability classifications and summaries has been developed, along with their applications and practical use cases. We believe this study will make it easier for researchers to understand all explainability methods and access their applications simultaneously.

show abstract

“…The goal of adversarial training is to incorporate the adversarial search within the training process and, thus, realize robustness against adversarial examples at test time. In particular, recently, Bayesian adversarial learning has been investigated and adopted in the computer vision domain to propose to improve the robustness of models against adversarial examples (Liu and Wang 2016;Ye and Zhu 2018;Liu et al 2019;Wicker et al 2021;Carbone et al 2020;Doan et al 2022).…”

Section: Background and Related Workmentioning

confidence: 99%

“…To construct a formulation to improve the robustness against feature-space adversarial malware examples, and ultimately problem space malware, we propose a Bayesian formulation for adversarially training a neural network: i) with the capability to capture the distribution of models to improve robustness (Liu and Wang 2016;Liu et al 2019;Ye and Zhu 2018;Wicker et al 2021;Carbone et al 2020;Doan et al 2022); and ii) prove our proposed method of diversified Bayesian neural networks hardened with adversarial training bounds the difference between the adversarial risk and the conventional empirical risk to theoretically explain the improved robustness.…”

Section: Introductionmentioning

confidence: 99%

Feature-Space Bayesian Adversarial Learning Improved Malware Detector Robustness

Doan,

Yang,

Montague

et al. 2023

AAAI

View full text Add to dashboard Cite

We present a new algorithm to train a robust malware detector. Malware is a prolific problem and malware detectors are a front-line defense. Modern detectors rely on machine learning algorithms. Now, the adversarial objective is to devise alterations to the malware code to decrease the chance of being detected whilst preserving the functionality and realism of the malware. Adversarial learning is effective in improving robustness but generating functional and realistic adversarial malware samples is non-trivial. Because: i) in contrast to tasks capable of using gradient-based feedback, adversarial learning in a domain without a differentiable mapping function from the problem space (malware code inputs) to the feature space is hard; and ii) it is difficult to ensure the adversarial malware is realistic and functional. This presents a challenge for developing scalable adversarial machine learning algorithms for large datasets at a production or commercial scale to realize robust malware detectors. We propose an alternative; perform adversarial learning in the feature space in contrast to the problem space. We prove the projection of perturbed, yet valid malware, in the problem space into feature space will always be a subset of adversarials generated in the feature space. Hence, by generating a robust network against feature-space adversarial examples, we inherently achieve robustness against problem-space adversarial examples. We formulate a Bayesian adversarial learning objective that captures the distribution of models for improved robustness. To explain the robustness of the Bayesian adversarial learning algorithm, we prove that our learning method bounds the difference between the adversarial risk and empirical risk and improves robustness. We show that Bayesian neural networks (BNNs) achieve state-of-the-art results; especially in the False Positive Rate (FPR) regime. Adversarially trained BNNs achieve state-of-the-art robustness. Notably, adversarially trained BNNs are robust against stronger attacks with larger attack budgets by a margin of up to 15% on a recent production-scale malware dataset of more than 20 million samples. Importantly, our efforts create a benchmark for future defenses in the malware domain.

show abstract

Resilience of Bayesian Layer-Wise Explanations under Adversarial Attacks

Cited by 5 publications

References 19 publications

An Introduction to Adversarially Robust Deep Learning

An Introduction to Adversarially Robust Deep Learning

An Empirical Survey on Explainable AI Technologies: Recent Trends, Use-Cases, and Categories from Technical and Application Perspectives

Feature-Space Bayesian Adversarial Learning Improved Malware Detector Robustness

Contact Info

Product

Resources

About