Reliably fast adversarial training via latent adversarial perturbation

Park, Geon Yeong; Lee, Sang Wan

doi:10.1109/iccv48922.2021.00766

Cited by 14 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bilateral Adversarial Training (BAT) [23] is designed to perturb both the image and the label during training. Many other methods are also proposed to accelerate the training of PGD-AT, such as Free [69], SLAT [70],…”

Section: Variants Of Pgd-at Many Researchers Have Proposed Various Me...mentioning

confidence: 99%

“…White-box L-BFGS [5] Optimisation-based Early attack using a box-constrained optimization method AMDR [41] Close distance between the input and the target-class input in latent space DeepFool [42] Estimate the minimum distance between inputs and decsion boundary C&W [20] Powerful empirically-chosen loss function to approximate, an optimisation problem similar to L-BFGS FGSM [6] Gradient-based Find perturbations direction fast with gradient ascent BIM [43] Multi-step variants of FGSM MI-FGSM [45] BIM with Momentum, faster to converge R+FGSM [46] Randomized initialized FGSM which helps to escape from local optimum PGD [37] Randomized initialized BIM, powerful attack to evaluate robustness BPDA [47] Approximation-based Replace non-differentiable parts with differentiable parts, to overcome gradient masking SPSA [48] gradient estimation method to overcome gradient masking ATN [52] Generate adversarial perturbation with neural networks AdvGAN [53] Generate adversarial perturbation with GAN Black-box SBA [55] Generate transfer attack on a substitute model that imitates the target model Zoo [56] Approximate the gradients of the objective function using finite-difference numerical estimates similar to SPSA OPA [60] Deceive the target model by only one pixel BA [61] Start from one existing adversarial sample and randomly walk to search the decision boundary NAA [ Train a model with the triplet loss ANL [22] Inject adversarial perturbation into latent features BAT [23] Perturb both the image and the label during training EAT [73] Train a model with adversarial examples generated on other pre-trained models PED [74] Force non-maximal predictions as diverse as possible in an ensemble system ALP [21] Force adversarial examples and their corresponding natural samples to have similar output FAT [76] Train a model with friendly adversarial examples which do not cross the decision boundary too much Overfitting [81] leverage early stop to choose the best checkpoint to inference Free [69] Accelerate AT by recycling the gradient information SLAT [70] Accelerate AT with the single-step laten...…”

Section: Remarksmentioning

confidence: 99%

See 1 more Smart Citation

A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies

Zhang¹,

Huang²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

In the last a few decades, deep neural networks have achieved remarkable success in machine learning, computer vision, and pattern recognition. Recent studies however show that neural networks (both shallow and deep) may be easily fooled by certain imperceptibly perturbed input samples called adversarial examples. Such security vulnerability has resulted in a large body of research in recent years because real-world threats could be introduced due to vast applications of neural networks. To address the robustness issue to adversarial examples particularly in pattern recognition, robust adversarial training has become one mainstream. Various ideas, methods, and applications have boomed in the field. Yet, a deep understanding of adversarial training including characteristics, interpretations, theories, and connections among different models has still remained elusive. In this paper, we present a comprehensive survey trying to offer a systematic and structured investigation on robust adversarial training in pattern recognition. We start with fundamentals including definition, notations, and properties of adversarial examples. We then introduce a unified theoretical framework for defending against adversarial samples -robust adversarial training with visualizations and interpretations on why adversarial training can lead to model robustness. Connections will be also established between adversarial training and other traditional learning theories. After that,

show abstract

Section: Variants Of Pgd-at Many Researchers Have Proposed Various Me...mentioning

confidence: 99%

Section: Remarksmentioning

confidence: 99%

A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies

Zhang¹,

Huang²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…robustness of neural networks, it requires more time to compute gradients of the neural network's input multiple times (Park and Lee 2021;Yu and Sun 2022;Izmailov et al 2018;Zhang et al 2020). As a result, single-step AT methods gain significant attention as a research hotspot due to their effectiveness and efficiency (Phan et al 2023;Chiang, Chan, and Wu 2021;Qin et al 2023).…”

Section: Introductionmentioning

confidence: 99%

Taxonomy Driven Fast Adversarial Training

Tong,

Jiang,

Gui

et al. 2024

AAAI

View full text Add to dashboard Cite

Adversarial training (AT) is an effective defense method against gradient-based attacks to enhance the robustness of neural networks. Among them, single-step AT has emerged as a hotspot topic due to its simplicity and efficiency, requiring only one gradient propagation in generating adversarial examples. Nonetheless, the problem of catastrophic overfitting (CO) that causes training collapse remains poorly understood, and there exists a gap between the robust accuracy achieved through single- and multi-step AT. In this paper, we present a surprising finding that the taxonomy of adversarial examples reveals the truth of CO. Based on this conclusion, we propose taxonomy driven fast adversarial training (TDAT) which jointly optimizes learning objective, loss function, and initialization method, thereby can be regarded as a new paradigm of single-step AT. Compared with other fast AT methods, TDAT can boost the robustness of neural networks, alleviate the influence of misclassified examples, and prevent CO during the training process while requiring almost no additional computational and memory resources. Our method achieves robust accuracy improvement of 1.59%, 1.62%, 0.71%, and 1.26% on CIFAR-10, CIFAR-100, Tiny ImageNet, and ImageNet-100 datasets, when against projected gradient descent PGD10 attack with perturbation budget 8/255. Furthermore, our proposed method also achieves state-of-the-art robust accuracy against other attacks. Code is available at https://github.com/bookman233/TDAT.

show abstract

“…Due to their vulnerability to artificial small perturbations, the adversarial robustness of deep neural networks has received great attention [40,13]. A large amount of attack and defense strategies have been proposed for classification in past years [5,37,46,47,54,57,43,1,52,14,39,34]. As an extension of classification, semantic segmentation also suffers from adversarial examples [50,2].…”

Section: Introductionmentioning

confidence: 99%

SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustness

Gu¹,

Zhao²,

Tresp³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep neural network-based image classifications are vulnerable to adversarial perturbations. The image classifications can be easily fooled by adding artificial small and imperceptible perturbations to input images. As one of the most effective defense strategies, adversarial training was proposed to address the vulnerability of classification models, where the adversarial examples are created and injected into training data during training. The attack and defense of classification models have been intensively studied in past years. Semantic segmentation, as an extension of classifications, has also received great attention recently. Recent work shows a large number of attack iterations are required to create effective adversarial examples to fool segmentation models. The observation makes both robustness evaluation and adversarial training on segmentation models challenging. In this work, we propose an effective and efficient segmentation attack method, dubbed SegPGD. Besides, we provide a convergence analysis to show the proposed SegPGD can create more effective adversarial examples than PGD under the same number of attack iterations. Furthermore, we propose to apply our SegPGD as the underlying attack method for segmentation adversarial training. Since SegPGD can create more effective adversarial examples, the adversarial training with our SegPGD can boost the robustness of segmentation models. Our proposals are also verified with experiments on popular Segmentation model architectures and standard segmentation datasets.

show abstract

Reliably fast adversarial training via latent adversarial perturbation

Cited by 14 publications

References 10 publications

A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies

A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies

Taxonomy Driven Fast Adversarial Training

SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustness

Contact Info

Product

Resources

About