Certified Patch Robustness via Smoothed Vision Transformers

Salman, Hadi; Jain, Sameer; Wong, Eric; Mądry, Aleksander

doi:10.1109/cvpr52688.2022.01471

Cited by 27 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the practical implications of p robustness, there has been growing interest in studying broader threat models that allow for large, perceptible perturbations to images. Examples include robustness to spatial transformations [29,30] and adversarial patches [31,32,33,34,35]. The majority of the work in this category considers local or simple transformations that retain most of original pixel content.…”

Section: Related Workmentioning

confidence: 99%

Benchmarking Robustness to Adversarial Image Obfuscations

Stimberg¹,

Chakrabarti²,

Lu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Automated content filtering and moderation is an important tool that allows online platforms to build striving user communities that facilitate cooperation and prevent abuse. Unfortunately, resourceful actors try to bypass automated filters in a bid to post content that violate platform policies and codes of conduct. To reach this goal, these malicious actors may obfuscate policy violating images (e.g. overlay harmful images by carefully selected benign images or visual patterns) to prevent machine learning models from reaching the correct decision. In this paper, we invite researchers to tackle this specific issue and present a new image benchmark. This benchmark, based on IMAGENET, simulates the type of obfuscations created by malicious actors. It goes beyond IMAGENET-C and IMAGENET-C by proposing general, drastic, adversarial modifications that preserve the original content intent. It aims to tackle a more common adversarial threat than the one considered by pnorm bounded adversaries. We evaluate 33 pretrained models on the benchmark and train models with different augmentations, architectures and training methods on subsets of the obfuscations to measure generalization. We hope this benchmark will encourage researchers to test their models and methods and try to find new approaches that are more robust to these obfuscations.

show abstract

Section: Related Workmentioning

confidence: 99%

Benchmarking Robustness to Adversarial Image Obfuscations

Stimberg¹,

Chakrabarti²,

Lu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Most of theses defenses were also shown to be prone to adversarial attacks [29]. Most adversarial patch defenses for image classification and object-detection utilize multiple runs of occlusion and reclassification [30,31,32,33]. Most of these defenses have not been evaluated for physical attacks on face recognition systems.…”

Section: Defending Against Adversarial Examplesmentioning

confidence: 99%

“…In the future we want to evaluate the performance of Vision Transformers instead of CNNs as they can handle different image ablations better than CNNs [31]. We also want to investigate the performance if Generative Adversarial Networks (GANs) as proposed in [41] inpaint the removed areas.…”

Section: Conclusion and Futureworkmentioning

confidence: 99%

Robust face recognition: How much face Is needed?

Niklas¹

2022

View full text Add to dashboard Cite

Face recognition systems are used in high security applications for identification, authentication and authorization. Being robust, is essential, not only towards Adversarial Examples, but also towards occluding accessories, such as facial masks, which become particularly relevant through the COVID19 pandemic. We have identified three inconspicuous facial areas to wear adversarial examples to attack face recognition. These are the mouth-nose section, the forehead and the eye area. In this paper, we will address the question of how much of a face needs to be present for successful identification and whether removing the identified critical regions is a viable countermeasure against adversarial examples.

show abstract

“…Studying the robustness of ViT has recently attracted a growing interest [2,13,38,50,68]. It is critical to improve ViT's robustness in order to deploy them safely in the realworld.…”

Section: Introductionmentioning

confidence: 99%

Fairness-aware Vision Transformer via Debiased Self-Attention

Yao¹,

Li²,

Khanduri³

et al. 2023

Preprint

View full text Add to dashboard Cite

Vision Transformer (ViT) has recently gained significant interest in solving computer vision (CV) problems due to its capability of extracting informative features and modeling long-range dependencies through the self-attention mechanism. To fully realize the advantages of ViT in realworld applications, recent works have explored the trustworthiness of ViT, including its robustness and explainability. However, another desiderata, fairness has not yet been adequately addressed in the literature. We establish that the existing fairness-aware algorithms (primarily designed for CNNs) do not perform well on ViT. This necessitates the need for developing our novel framework via Debiased Self-Attention (DSA). DSA is a fairness-through-blindness approach that enforces ViT to eliminate spurious features correlated with the sensitive attributes for bias mitigation. Notably, adversarial examples are leveraged to locate and mask the spurious features in the input image patches. In addition, DSA utilizes an attention weights alignment regularizer in the training objective to encourage learning informative features for target prediction. Importantly, our DSA framework leads to improved fairness guarantees over prior works on multiple prediction tasks without compromising target prediction performance.

show abstract

Certified Patch Robustness via Smoothed Vision Transformers

Cited by 27 publications

References 20 publications

Benchmarking Robustness to Adversarial Image Obfuscations

Benchmarking Robustness to Adversarial Image Obfuscations

Robust face recognition: How much face Is needed?

Fairness-aware Vision Transformer via Debiased Self-Attention

Contact Info

Product

Resources

About