Adversarial Imaging Pipelines

Phan, Buu; Mannan, Fahim; Heide, Felix

doi:10.1109/cvpr46437.2021.01579

Cited by 22 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These methods typically assume that the camera acquisition and subsequent hardware processing do not alter the adversarial patterns. However, Phan et al [60] have recently realized attacks of individual camera types by exploiting slight differences in their hardware ISPs and optical systems.…”

Section: Adversarial Attack Methodsmentioning

confidence: 99%

All You Need is RAW: Defending Against Adversarial Attacks with Camera Image Pipelines

Zhang¹,

Dong²,

Heide³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Existing neural networks for computer vision tasks are vulnerable to adversarial attacks: adding imperceptible perturbations to the input images can fool these methods to make a false prediction on an image that was correctly predicted without the perturbation. Various defense methods have proposed image-to-image mapping methods, either including these perturbations in the training process or removing them in a preprocessing denoising step. In doing so, existing methods often ignore that the natural RGB images in today's datasets are not captured but, in fact, recovered from RAW color filter array captures that are subject to various degradations in the capture. In this work, we exploit this RAW data distribution as an empirical prior for adversarial defense. Specifically, we proposed a model-agnostic adversarial defensive method, which maps the input RGB images to Bayer RAW space and back to output RGB using a learned camera image signal processing (ISP) pipeline to eliminate potential adversarial patterns. The proposed method acts as an off-the-shelf preprocessing module and, unlike model-specific adversarial training methods, does not require adversarial images to train. As a result, the method generalizes to unseen tasks without additional retraining. Experiments on large-scale datasets (e.g., ImageNet, COCO) for different vision tasks (e.g., classification, semantic segmentation, object detection) validate that the method significantly outperforms existing methods across task domains.

show abstract

Section: Adversarial Attack Methodsmentioning

confidence: 99%

All You Need is RAW: Defending Against Adversarial Attacks with Camera Image Pipelines

Zhang¹,

Dong²,

Heide³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Unlike modifying the pixel of the RGB images, Phan et al [58] proposed to modify the RAW image that outputs from the camera device directly, which makes it robust to compression (i.e., JPEG). They demonstrated that perturbations devised for RAW images remain physically aggressive when printed out and retaken by the camera.…”

Section: Attack Imaging Pipelinesmentioning

confidence: 99%

A Survey on Physical Adversarial Attack in Computer Vision

Wang¹,

Wen²,

Jiang³

et al. 2023

Preprint

View full text Add to dashboard Cite

In the past decade, deep learning has dramatically changed the traditional hand-craft feature manner with strong feature learning capability, promoting the tremendous improvement of conventional tasks. However, deep neural networks (DNNs) have been demonstrated to be vulnerable to adversarial examples crafted by small noise, which is imperceptible to human observers but can make DNNs misbehave. Existing adversarial attacks can be divided into digital and physical adversarial attacks. The former is designed to pursue strong attack performance in lab environments while hardly remaining effective when applied to the physical world. In contrast, the latter focus on developing physical deployable attacks, which are more robust in complex physical environmental conditions (e.g., brightness, occlusion, etc.). Recently, with the increasing deployment of the DNN-based system in the real world, enhancing the robustness of these systems is an emergency, while exploring physical adversarial attacks exhaustly is the precondition. To this end, this paper reviews the development of physical adversarial attacks against DNN-based computer vision tasks (i.e., image recognition and object detection tasks), which can provide beneficial information for developing stronger physical adversarial attacks. For completeness, we also briefly introduce the works that do not involve physical attacks but are closely related to them. Specifically, we first proposed a taxonomy to summarize the current physical adversarial attacks. Then we briefly discuss the existing physical attacks and focus on the technique for improving the robustness of physical attacks under complex physical environmental conditions. Finally, we discuss the issues of the current physical adversarial attacks to be solved and give promising directions.

show abstract

“…Although the prior work [11] designed the adversarial attacks against ISP procedure, it assumed that deep models directly consume the obtained adversarial images without considering the preprocessing process. The preprocessing operation, specifically image scaling, can destroy adversarial patterns of attack images [12].…”

Section: Existing Attacks Potential Attacksmentioning

confidence: 99%

Adversarial RAW: Image-Scaling Attack Against Imaging Pipeline

Li¹,

Chen²

2022

Preprint

View full text Add to dashboard Cite

Deep learning technologies have become the backbone for the development of computer vision. With further explorations, deep neural networks have been found vulnerable to well-designed adversarial attacks. Most of the vision devices are equipped with image signal processing (ISP) pipeline to implement RAW-to-RGB transformations and embedded into data preprocessing module for efficient image processing. Actually, ISP pipeline can introduce adversarial behaviors to post-capture images while data preprocessing may destroy attack patterns. However, none of the existing adversarial attacks takes into account the impacts of both ISP pipeline and data preprocessing.In this paper, we develop an image-scaling attack targeting on ISP pipeline, where the crafted adversarial RAW can be transformed into attack image that presents entirely different appearance once being scaled to a specific-size image. We first consider the gradient-available ISP pipeline, i.e., the gradient information can be directly used in the generation process of adversarial RAW to launch the attack. To make the adversarial attack more applicable, we further consider the gradient-unavailable ISP pipeline, in which a proxy model that well learns the RAWto-RGB transformations is proposed as the gradient oracles. Extensive experiments show that the proposed adversarial attacks can craft adversarial RAW data against the target ISP pipelines with high attack rates.

show abstract

Adversarial Imaging Pipelines

Cited by 22 publications

References 27 publications

All You Need is RAW: Defending Against Adversarial Attacks with Camera Image Pipelines

All You Need is RAW: Defending Against Adversarial Attacks with Camera Image Pipelines

A Survey on Physical Adversarial Attack in Computer Vision

Adversarial RAW: Image-Scaling Attack Against Imaging Pipeline

Contact Info

Product

Resources

About