Adversarial examples for generative models

Kos, Jernej; Fischer, Ian; Song, Dawn

doi:10.48550/arxiv.1702.06832

Cited by 22 publications

(19 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…these successes, they are increasingly being used as part of control pipelines in physical systems such as cars [8,17], UAVs [4,24], and robots [40]. Recent work, however, has demonstrated that DNNs are vulnerable to adversarial perturbations [5,9,10,15,16,22,25,29,30,35]. These carefully crafted modifications to the (visual) input of DNNs can cause the systems they control to misbehave in unexpected and potentially dangerous ways.…”

Section: Introductionmentioning

confidence: 99%

Robust Physical-World Attacks on Deep Learning Visual Classification

Eykholt

Evtimov

Fernandes

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

1,756

1,143

View full text Add to dashboard Cite

Recent studies show that the state-of-the-art deep neural networks (DNNs) are vulnerable to adversarial examples, resulting from small-magnitude perturbations added to the input. Given that that emerging physical systems are using DNNs in safety-critical situations, adversarial examples could mislead these systems and cause dangerous situations. Therefore, understanding adversarial examples in the physical world is an important step towards developing resilient learning algorithms. We propose a general attack algorithm, Robust Physical Perturbations (RP 2 ), to generate robust visual adversarial perturbations under different physical conditions. Using the real-world case of road sign classification, we show that adversarial examples generated using RP 2 achieve high targeted misclassification rates against standard-architecture road sign classifiers in the physical world under various environmental conditions, including viewpoints. Due to the current lack of a standardized testing method, we propose a two-stage evaluation methodology for robust physical adversarial examples consisting of lab and field tests. Using this methodology, we evaluate the efficacy of physical adversarial manipulations on real objects. With a perturbation in the form of only black and white stickers, we attack a real stop sign, causing targeted misclassification in 100% of the images obtained in lab settings, and in 84.8% of the captured video frames obtained on a moving vehicle (field test) for the target classifier.

show abstract

Section: Introductionmentioning

confidence: 99%

Robust Physical-World Attacks on Deep Learning Visual Classification

Eykholt

Evtimov

Fernandes

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

1,756

1,143

View full text Add to dashboard Cite

show abstract

“…Different attacks against machine learning models have been proposed in recent years. For example, reverse engineering attacks [17,36] steal model parameters and structures; adversarial learning [13,22,27,35] generates misleading examples that will be misclassified by the model; model inversion attacks [10,14] infer the features of a record based on the model's predictions on it; membership inference attacks [32] infer the presence of a record in the model's training dataset.…”

Section: Related Workmentioning

confidence: 99%

Understanding Membership Inferences on Well-Generalized Learning Models

Long¹,

Bindschaedler²,

Wang³

et al. 2018

Preprint

View full text Add to dashboard Cite

Membership Inference Attack (MIA) determines the presence of a record in a machine learning model's training data by querying the model. Prior work has shown that the attack is feasible when the model is overfitted to its training data or when the adversary controls the training algorithm. However, when the model is not overfitted and the adversary does not control the training algorithm, the threat is not well understood. In this paper, we report a study that discovers overfitting to be a sufficient but not a necessary condition for an MIA to succeed. More specifically, we demonstrate that even a well-generalized model contains vulnerable instances subject to a new generalized MIA (GMIA). In GMIA, we use novel techniques for selecting vulnerable instances and detecting their subtle influences ignored by overfitting metrics. Specifically, we successfully identify individual records with high precision in real-world datasets by querying black-box machine learning models. Further we show that a vulnerable record can even be indirectly attacked by querying other related records and existing generalization techniques are found to be less effective in protecting the vulnerable instances. Our findings sharpen the understanding of the fundamental cause of the problem: the unique influences the training instance may have on the model.

show abstract

“…Deep neural networks (DNNs) are widely applied in computer vision, natural language, and robotics, espe-cially in safety-critical tasks such as autonomous driving [10]. At the same time, DNNs have been shown to be vulnerable to adversarial examples [3,7,8,15,18], maliciously perturbed inputs that cause DNNs to produce incorrect predictions. These attacks pose a risk to the use of deep learning in safety-and security-critical decisions.…”

Section: Introductionmentioning

confidence: 99%

Physical Adversarial Examples for Object Detectors

Song¹,

Eykholt²,

Evtimov³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

Deep neural networks (DNNs) are vulnerable to adversarial examples-maliciously crafted inputs that cause DNNs to make incorrect predictions. Recent work has shown that these attacks generalize to the physical domain, to create perturbations on physical objects that fool image classifiers under a variety of real-world conditions. Such attacks pose a risk to deep learning models used in safety-critical cyber-physical systems.In this work, we extend physical attacks to more challenging object detection models, a broader class of deep learning algorithms widely used to detect and label multiple objects within a scene. Improving upon a previous physical attack on image classifiers, we create perturbed physical objects that are either ignored or mislabeled by object detection models. We implement a Disappearance Attack, in which we cause a Stop sign to "disappear" according to the detector-either by covering the sign with an adversarial Stop sign poster, or by adding adversarial stickers onto the sign. In a video recorded in a controlled lab environment, the state-of-the-art YOLO v2 detector failed to recognize these adversarial Stop signs in over 85% of the video frames. In an outdoor experiment, YOLO was fooled by the poster and sticker attacks in 72.5% and 63.5% of the video frames respectively. We also use Faster R-CNN, a different object detection model, to demonstrate the transferability of our adversarial perturbations. The created poster perturbation is able to fool Faster R-CNN in 85.9% of the video frames in a controlled lab environment, and 40.2% of the video frames in an outdoor environment. Finally, we present preliminary results with a new Creation Attack, wherein innocuous physical stickers fool a model into detecting nonexistent objects.

show abstract

Adversarial examples for generative models

Cited by 22 publications

References 6 publications

Robust Physical-World Attacks on Deep Learning Visual Classification

Robust Physical-World Attacks on Deep Learning Visual Classification

Understanding Membership Inferences on Well-Generalized Learning Models

Physical Adversarial Examples for Object Detectors

Contact Info

Product

Resources

About