Adversarial examples in the physical world

Abstract

Most existing machine learning classifiers are highly vulnerable to adversarial examples. An adversarial example is a sample of input data which has been modified very slightly in a way that is intended to cause a machine learning classifier to misclassify it. In many cases, these modifications can be so subtle that a human observer does not even notice the modification at all, yet the classifier still makes a mistake. Adversarial examples pose security concerns because they could be used to perform an attack on machine learning systems, even if the adversary has no access to the underlying model. Up to now, all previous work has assumed a threat model in which the adversary can feed data directly into the machine learning classifier. This is not always the case for systems operating in the physical world, for example those which are using signals from cameras and other sensors as input. This paper shows that even in such physical world scenarios, machine learning systems are vulnerable to adversarial examples. We demonstrate this by feeding adversarial images obtained from a cell-phone camera to an ImageNet Inception classifier and measuring the classification accuracy of the system. We find that a large fraction of adversarial examples are classified incorrectly even when perceived through the camera.

Adversarially Robust Stability Certificates can be Sample-Efficient

“…Adversarial robustness Traditional approaches [36,22,46,16] to adversarial learning consider worstcase perturbations to the data during training, i.e., the data is perturbed after it has been generated. While such a perturbation model is meaningful in the image classification setting for which adversarial robust training methods were originally developed, it does not immediately translate to the dynamic setting that we consider, where the adversary may be used to capture model uncertainty or process noise.…”

Wiggling Weights to Improve the Robustness of Classifiers

“…adversarial perturbations to the images it is possible to change the prediction of the classifier. Since then, plenty of research [4,25,28,29,39] has been performed on finding different types of adversarial perturbations and study the robustification against them [4,10,15,15,25]. In this work, we utilize a strong yet undefended attack basic iterative method [25] for generating adversarial perturbations.…”

“…Since then, plenty of research [4,25,28,29,39] has been performed on finding different types of adversarial perturbations and study the robustification against them [4,10,15,15,25]. In this work, we utilize a strong yet undefended attack basic iterative method [25] for generating adversarial perturbations. [32] focused on robustification against adversarial as well as natural perturbations by using properly tuned Gaussian and Speckle noise.…”

LTD: Low Temperature Distillation for Robust Adversarial Training

“…Such kinds of attacks can also exist for audio space [9] and text classification [10] in natural language processing. Those attacks exist not only in the digital world, but can also happen in the physical world, such as the cellphone camera attack [11] or road sign attack [12]. Besides the inference time attacks, the backdoor attacks [13], [14] that weaken model accuracy during the training time have also been investigated.…”