Universal Decision-Based Black-Box Perturbations: Breaking Security-Through-Obscurity Defenses

Hogan, Thomas A.; Kailkhura, Bhavya

doi:10.48550/arxiv.1811.03733

Cited by 3 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, a model's confidence score alone may not be very reliable. For example, in computer vision, well-crafted perturbations to images can cause classifiers to make mistakes (such as, identifying a panda as a gibbon or confusing a cat with a computer) with very high confidence 5 . As we will show later, this problem also persists in the Materials Informatics pipeline (especially with distributional skewness).…”

Section: Better Evaluation and Uncertainty Quantification Techniques ...mentioning

confidence: 99%

Reliable and explainable machine-learning methods for accelerated material discovery

et al. 2019

Self Cite

View full text Add to dashboard Cite

Material scientists are increasingly adopting the use of machine learning (ML) for making potentially important decisions, such as, discovery, development, optimization, synthesis and characterization of materials. However, despite ML's impressive performance in commercial applications, several unique challenges exist when applying ML in materials science applications. In such a context, the contributions of this work are twofold. First, we identify common pitfalls of existing ML techniques when learning from underrepresented/imbalanced material data. Specifically, we show that with imbalanced data, standard methods for assessing quality of ML models break down and lead to misleading conclusions. Furthermore, we find that the model's own confidence score cannot be trusted and model introspection methods (using simpler models) do not help as they result in loss of predictive performance (reliability-explainability trade-off). Second, to overcome these challenges, we propose a general-purpose explainable and reliable machine-learning framework. Specifically, we propose a novel pipeline that employs an ensemble of simpler models to reliably predict material properties. We also propose a transfer learning technique and show that the performance loss due to models' simplicity can be overcome by exploiting correlations among different material properties. A new evaluation metric and a trust score to better quantify the confidence in the predictions are also proposed. To improve the interpretability, we add a rationale generator component to our framework which provides both model-level and decision-level explanations. Finally, we demonstrate the versatility of

show abstract

Section: Better Evaluation and Uncertainty Quantification Techniques ...mentioning

confidence: 99%

Reliable and explainable machine-learning methods for accelerated material discovery

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Universal perturbations: MimicGAN also provides effective defense against universal perturbations [41,3], which belong to the class of image-agnostic perturbations where an attack is just a single vector which when added to the entire dataset can fool a classifier. To test this defense, we first design a targeted universal perturbation using the Fast Gradient Sign Method (FGSM) [21], by computing the mean adversarial perturbation from N = 15 test images, i.e.…”

Section: Adversarial Defensementioning

confidence: 99%

MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking

2020

Self Cite

View full text Add to dashboard Cite

In the past few years, Generative Adversarial Networks (GANs) have dramatically advanced our ability to represent and parameterize high-dimensional, non-linear image manifolds. As a result, they have been widely adopted across a variety of applications, ranging from challenging inverse problems like image completion, to problems such as anomaly detection and adversarial defense. A recurring theme in many of these applications is the notion of projecting an image observation onto the manifold that is inferred by the generator. In this context, Projected Gradient Descent (PGD) has been the most popular approach, which essentially optimizes for a latent vector that minimizes the discrepancy between a generated image and the given observation. However, PGD is a brittle optimization technique that fails to identify the right projection (or latent vector) when the observation is corrupted, or perturbed even by a small amount. Such corruptions are common in the real world, for example images in the wild come with unknown crops, rotations, missing pixels, or other kinds of non-linear distributional shifts which break current encoding methods, rendering downstream applications unusable. To address this, we propose corruption mimicking -a new robust projection technique, that utilizes a surrogate network to approximate the unknown corruption directly at test time, without the need for additional supervision or data augmentation. The proposed method is significantly more robust than PGD and other competing methods under a wide variety of corruptions, thereby enabling a more effective use of GANs in real-world applications. More importantly, we show that our approach produces state-of-the-art performance in several GAN-based applications -anomaly detection, domain adaptation, and adversarial defense, that benefit from an accurate projection.

show abstract

“…(a) Universal Perturbations [19] Attack No Defense Cowboy [26] Defense GAN [25] MimicGAN (Ours) DeepFool [20] Universal perturbations: MimicGAN provides a natural defense against universal perturbations [19,1], which are a class of image-agnostic perturbations where an attack is just a single vector which when added to the entire dataset can fool a classifier. To test this defense, we first design a targeted universal perturbation using the Fast Gradient Sign Method (FGSM) [11], by computing the mean adversarial perturbation from N = 15 test images, i.e.…”

Section: Adversarial Defensementioning

confidence: 99%

MimicGAN: Corruption-Mimicking for Blind Image Recovery & Adversarial Defense

Anirudh,

Thiagarajan,

Kailkhura

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Universal Decision-Based Black-Box Perturbations: Breaking Security-Through-Obscurity Defenses

Cited by 3 publications

References 12 publications

Reliable and explainable machine-learning methods for accelerated material discovery

Reliable and explainable machine-learning methods for accelerated material discovery

MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking

MimicGAN: Corruption-Mimicking for Blind Image Recovery & Adversarial Defense

Contact Info

Product

Resources

About