Adversarial Attacks Beyond the Image Space

Abstract

Generating adversarial examples is an intriguing problem and an important way of understanding the working mechanism of deep neural networks. Most existing approaches generated perturbations in the image space, i.e., each pixel can be modified independently. However, in this paper we pay special attention to the subset of adversarial examples that correspond to meaningful changes in 3D physical properties (like rotation and translation, illumination condition, etc.). These adversaries arguably pose a more serious concern, as they demonstrate the possibility of causing neural network failure by easy perturbations of real-world 3D objects and scenes.In the contexts of object classification and visual question answering, we augment state-of-the-art deep neural networks that receive 2D input images with a rendering module (either differentiable or not) in front, so that a 3D scene (in the physical space) is rendered into a 2D image (in the image space), and then mapped to a prediction (in the output space). The adversarial perturbations can now go beyond the image space, and have clear meanings in the 3D physical world. Though image-space adversaries can be interpreted as per-pixel albedo change, we verify that they cannot be well explained along these physically meaningful dimensions, which often have a non-local effect. But it is still possible to successfully attack beyond the image space on the physical space, though this is more difficult than image-space attacks, reflected in lower success rates and heavier perturbations required. 3D Object 2D ImageRound #1: car  Round #2: car  Round #T: bus  …… Gradient Back-Prop rendering CNN modifying 2D image modifying 3D scene Beyond the Image Space In the Image Space Attack Success! arXiv:1711.07183v6 [cs.CV] Differentiable Attacks Non-differentiable Attacks R: bench  R: chair  = 3.7 × 10 −3 conf = 89.9% = 4.7 × 10 −3 R: table  conf = 89.9% Image Space Physical Space Original Input Image R: cap  R: helmet  Physical Space Original Input Image Physical-Space Attack DetailsRotating the object by −2.9, 9.4 and 2.5 (× 10 −3 rad) by , and axes; then moving it by 2.0, 0.0 and 0.2 (× 10 −3 unit length) along , and axes; tuning its color by 9.1, 5.4 and −4.8 (× 10 −2 max intensity) in the RGB space; adjusting the light source by −0.3 unit; and change the light angle by 9.5, 5.4 and 0.6 (× 10 −2 unit).

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

“…However, in a follow-up work Lu et al [19] showed that the detectors like YOLO 9000 [149] and FasterRCNN [150] are 'currently' not fooled by the attacks introduced by Etimov et al [75]. Zeng et al [87] also argue that adversarial perturbations in the image space do not generalize well in the physical space Fig. 7: Example of road sign attack [75]: The success rate of fooling LISA-CNN [75] classifier on all the shown images is 100%.…”

Differentiable Monte Carlo ray tracing through edge sampling

“…Our system can be used for mining adversarial examples of 3D scenes, since it provides the ability to backpropagate from image to scene parameters. A similar idea has been explored by Zeng et al [2017], but we use a more general renderer. We demonstrate this in Figure 10.…”

MeshAdv: Adversarial Meshes for Visual Recognition

“…Beyond perturbations in texture form, Zeng et al [60] perturbed the physical parameters (normal, illumination and material) for untargeted attacks against 3D shape classification and a VQA system. However, for the differentiable renderer, they assume that the camera parameters are known beforehand and then perturb 2D normal maps under the fixed projection.…”

“…This means the manipulation space can be largely reduced due to image parameterization. 2) Constraints in 3D: 3D constraints such as physically possible shape geometry and texture are not directly reflected on 2D [60]. Human perception of an object are in 3D or 2.5D [34], and perturbation of shape or texture on 3D objects may directly affect human perception of them.…”