Understanding Robustness of Transformers for Image Classification

Artificial Intelligence (AI) can support diagnostic workflows in oncology by aiding diagnosis and providing biomarkers. AI applications are therefore expected to evolve from academic prototypes to commercial products in the coming years. However, AI applications are vulnerable to adversarial attacks, such as malicious interference with test data aiming to cause misclassifications. Therefore, it is essential for the use of AI-based diagnostic devices to secure them against such attacks before widespread use. Unfortunately, no resistant systems exist in computational pathology so far. To address this problem, we investigate the susceptibility of convolutional neural networks (CNNs) to multiple types of white- and black-box attacks. We demonstrate that both attacks can easily confuse CNNs in clinically relevant pathology tasks and impair classification performance. Classical adversarially robust training and dual batch normalization (DBN) are possible mitigation strategies but require precise knowledge of the type of attack used in the inference. We demonstrate that vision transformers (ViTs) perform equally well compared to CNNs at baseline and are orders of magnitude more robust to different types of white-box and black-box attacks. At a mechanistic level, we show that this is associated with a more robust latent representation of clinically relevant categories in ViTs compared to CNNs. Our results are in line with previous theoretical studies. We show that ViTs are robust learners in computational pathology. This implies that large-scale rollout of AI models in computational pathology should rely on ViTs rather than CNN-based classifiers to provide inherent protection against adversaries.

show abstract

“…Also, our findings are in line with studies in non-medical domains which analyzed robustness of ViTs in technical benchmark tasks. 50,51…”

Section: Discussionmentioning

confidence: 99%

Adversarial attacks and adversarial robustness in computational pathology

Laleh

Truhn

Veldhuizen

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Most existing decision-based attack algorithms are dense attacks (the objective is to minimise L 2 or L ∞ distortion). Interestingly, these methods, including BA (Brendel et al, 2018), HSJA (Chen et al, 2020), QEBA (Li et al, 2020), NLBA (Li et al, 2021), PSBA (Zhang et al, 2021), Sign- OPT Cheng et al (2020) or the covariance matrix adaptation evolution strategy (CMA-ES) based method for face recognition tasks in (Dong et al, 2019), can be adapted to a sparse attack setting by a projection to L 0 -ball; however this is not effective, as we show later in Appendix A.7. Although CMA-ES (Dong et al, 2019) is an evolutionary algorithm, albeit for a dense attack, the formulation requires individuals of a population to be real number vectors that can be sampled from a Gaussian distribution.…”

Section: White-box Methodsmentioning

confidence: 99%

“…In contrast, Pointwise employs a purely random method to select the pixel dimension and position i, j to alter. We are motivated to investigate if decision-based dense attacks (L 2 and L ∞ constrained) such as BA (Brendel et al, 2018), HSJA (Chen et al, 2020), QEBA (Li et al, 2020), NLBA (Li et al, 2021), PSBA (Zhang et al, 2021), SignOPT (Cheng et al, 2020) or RayS (Chen & Gu, 2020) can be adapted to a sparse setting by a projection to L 0 -ball. This idea is promising because PGD can be successfully adapted to a sparse setting to provide a sparse attack algorithm in a whitebox setting.…”

Section: A4 Robustness Of Sparse Attacks Against An Adversarially Tra...mentioning

confidence: 99%

“…Scope of the Study-Vision Transformers and Convolutional Networks. Attention-based architectures introduced by Cordonnier et al (2020); Ramachandran et al (2019); Touvron et al (2021), particularly the Vision Transformer (ViT) model proposed by Dosovitskiy et al (2021), can be competitive or even outperform convolution-based architectures (Bhojanapalli et al, 2021;Carion et al, 2020). Existing studies have not considered adversarial attacks in l 0 norm constraint based perturbation regimes aginst ViT, although a few studies have explored robustness against l 2 and l ∞ norm constraints (Shao et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Query Efficient Decision Based Sparse Attacks Against Black-Box Deep Learning Models

Vo¹,

Abbasnejad²,

Ranasinghe³

2022

Preprint

View full text Add to dashboard Cite

Despite our best efforts, deep learning models remain highly vulnerable to even tiny adversarial perturbations applied to the inputs. The ability to extract information form solely the output of a machine learning model to craft adversarial perturbations to black-box models is a practical threat against real-world systems, such as autonomous cars or machine learning models exposed as a service (MLaaS). Of particular interest are sparse attacks. The realisation of sparse attacks in blackbox models demonstrates that machine learning models are more vulnerable than we believe. Because, these attacks aim to minimize the number of perturbed pixels-measured by l 0 norm-required to mislead a model by solely observing the decision (the predicted label) returned to a model query; the so-called decisionbased attack setting. But, such an attack leads to an NP-hard optimization problem. We develop an evolution-based algorithm-SparseEvo-for the problem and evaluate against both convolutional deep neural networks and vision transformers. Notably, vision transformers are yet to be investigated under a decision-based attack setting. SparseEvo requires significantly fewer model queries than the stateof-the-art sparse attack Pointwise for both untargeted and targeted attacks. The attack algorithm, although conceptually simple, is also competitive with only a limited query budget against the state-of-the-art gradient-based whitebox attacks in standard computer vision tasks such as ImageNet. Importantly, the query efficient SparseEvo, along with decision-based attacks, in general, raise new questions regarding the safety of deployed systems and poses new directions to study and understand the robustness of machine learning models.

show abstract

“…Investigating the robustness of pre-trained ViT is an active research area and there are several concurrent papers exploring robustness of ViT to adversarial perturbations and common corruptions (ImageNet-C). Bhojanapalli et al [2021] show the robustness of pre-trained transformers to input perturbations, and of transformers to layer removal. Caron et al [2021] demonstrate many emerging properties in self-supervised ViTs, while Shao et al…”

Section: Pre-training Neural Networkmentioning

confidence: 99%

Exploring the Limits of Out-of-Distribution Detection

Fort,

Ren,

Lakshminarayanan

2021

Preprint

View full text Add to dashboard Cite

Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers can significantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 vs CIFAR-10 OOD detection, we improve the AUROC from 85% (current SOTA) to more than 96% using Vision Transformers pre-trained on ImageNet-21k. On a challenging genomics OOD detection benchmark, we improve the AUROC from 66% to 77% using transformers and unsupervised pre-training. To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that pre-trained transformers are particularly well-suited for outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 can be improved to 98.7% with just 1 image per OOD class, and 99.46% with 10 images per OOD class. For multi-modal image-text pre-trained transformers such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information without any accompanying images, and show that this outperforms previous SOTA on standard vision OOD benchmark tasks. * Equal contribution.Preprint. Under review.

show abstract

Understanding Robustness of Transformers for Image Classification

Cited by 275 publications

References 15 publications

Adversarial attacks and adversarial robustness in computational pathology

Adversarial attacks and adversarial robustness in computational pathology

Query Efficient Decision Based Sparse Attacks Against Black-Box Deep Learning Models

Exploring the Limits of Out-of-Distribution Detection

Contact Info

Product

Resources

About