MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius

Zhai, Runtian; Chen, Dan; He, Di; Zhang, Huan; Gong, Boqing; Ravikumar, Pradeep; Hsieh, Cho‐Jui; Wang, Liwei

doi:10.48550/arxiv.2001.02378

Cited by 24 publications

(31 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The average certified radius of these customized smoothed classifiers g(f, x i , σ i ) is higher than that of the existing RS via uniform noise across examples. Insta-RS significantly improves the average certified radius (ACR) of the state-of-the-art pretrained Cohen et al (2019) model by 14.23%, Zhai et al (2020) model by 5.17% and Salman et al (2019) model by 3.08% on Cifar-10. On ImageNet, Insta-RS boosts the ACR of two pre-trained models from (Cohen et al, 2019) and (Salman et al, 2019) by 3.01% and 5.28% respectively.…”

Section: Summary Of Contributionsmentioning

confidence: 97%

“…best ACR at very different noise levels. For instance, the model from Zhai et al (2020) achieves the largest test ACR at σ 2 = 13 • 0.25 2 , very different from the model learned in Cohen et al (2019). Lastly, it is also important to see using a universal σ that is equal to what is used during training does not necessarily lead to the best robustness.…”

Section: Drawback Imentioning

confidence: 99%

“…Second, different models achieve the The average certified radius versus increasing levels of Gaussian noise on three pretrained models. The models used are from (Cohen et al, 2019), (Zhai et al, 2020) and (Salman et al, 2019), trained with σ = 0.5, σ = 1.0 and σ = 1.0 respectively. We test the three models on 500 test examples using 16 different values of the Gaussian variance:…”

Section: Drawback Imentioning

confidence: 99%

“…To address this challenge, we adopt an approximation for g(f, x i , σ 2 i ) and its certified radius to reduce the computational complexity of learning σ during test time. We resort to the soft version of g(f, x i , σ 2 i ) (Zhai et al, 2020):…”

Section: Insta-rs: Instance-wise Randomized Smoothingmentioning

confidence: 99%

“…Training of a base model f under a uniform Gaussian noise for all instances has been extensively studied in existing works by Cohen (Cohen et al, 2019), Macer (Zhai et al, 2020) and SmoothADV (Salman et al, 2019). Cohen (Cohen et al, 2019) proposes to use Gaussian augmentation during training and learn a base model f that is likely to perform well during test when smoothed with the same Gaussian distribution.…”

Section: Training Of Insta-rsmentioning

confidence: 99%

See 4 more Smart Citations

Insta-RS: Instance-wise Randomized Smoothing for Improved Robustness and Accuracy

Chen,

Kong,

et al. 2021

Preprint

View full text Add to dashboard Cite

Randomized smoothing (RS) is an effective and scalable technique for constructing neural network classifiers that are certifiably robust to adversarial perturbations. Most RS works focus on training a good base model that boosts the certified robustness of the smoothed model. However, existing RS techniques treat every data point the same, i.e., the variance of the Gaussian noise used to form the smoothed model is preset and universal for all training and test data. This preset and universal Gaussian noise variance is suboptimal since different data points have different margins and the local properties of the base model vary across the input examples. In this paper, we examine the impact of customized handling of examples and propose Instance-wise Randomized Smoothing (Insta-RS) -a multiple-start search algorithm that assigns customized Gaussian variances to test examples. We also design Insta-RS Train -a novel two-stage training algorithm that adaptively adjusts and customizes the noise level of each training example for training a base model that boosts the certified robustness of the instance-wise Gaussian smoothed model. Through extensive experiments on CIFAR-10 and ImageNet, we show that our method significantly enhances the average certified radius (ACR) as well as the clean data accuracy compared to existing state-of-theart provably robust classifiers.

show abstract

Section: Summary Of Contributionsmentioning

confidence: 97%

Section: Drawback Imentioning

confidence: 99%

Section: Drawback Imentioning

confidence: 99%

Section: Insta-rs: Instance-wise Randomized Smoothingmentioning

confidence: 99%

Section: Training Of Insta-rsmentioning

confidence: 99%

See 3 more Smart Citations

Insta-RS: Instance-wise Randomized Smoothing for Improved Robustness and Accuracy

Chen,

Kong,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Self-Progressing Robust Training

Cheng

Chen

Liu³

et al. 2021

AAAI

Self Cite

View full text Add to dashboard Cite

Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy machine learning systems. Current robust training methods such as adversarial training explicitly uses an ``attack'' (e.g., l_infty-norm bounded perturbation) to generate adversarial examples during model training for improving adversarial robustness. In this paper, we take a different perspective and propose a new framework SPROUT, self-progressing robust training. During model training, SPROUT progressively adjusts training label distribution via our proposed parametrized label smoothing technique, making training free of attack generation and more scalable. We also motivate SPROUT using a general formulation based on vicinity risk minimization, which includes many robust training methods as special cases. Compared with state-of-the-art adversarial training methods (PGD-l_infty and TRADES) under l_infty-norm bounded attacks and various invariance tests, SPROUT consistently attains superior performance and is more scalable to large neural networks. Our results shed new light on scalable, effective and attack-independent robust training methods.

show abstract

DeformRS: Certifying Input Deformations with Randomized Smoothing

Alfarra

Bibi

Khan

et al. 2022

AAAI

View full text Add to dashboard Cite

Deep neural networks are vulnerable to input deformations in the form of vector fields of pixel displacements and to other parameterized geometric deformations e.g. translations, rotations, etc. Current input deformation certification methods either (i) do not scale to deep networks on large input datasets, or (ii) can only certify a specific class of deformations, e.g. only rotations. We reformulate certification in randomized smoothing setting for both general vector field and parameterized deformations and propose DeformRS-VF and DeformRS-Par, respectively. Our new formulation scales to large networks on large input datasets. For instance, DeformRS-Par certifies rich deformations, covering translations, rotations, scaling, affine deformations, and other visually aligned deformations such as ones parameterized by Discrete-Cosine-Transform basis. Extensive experiments on MNIST, CIFAR10, and ImageNet show competitive performance of DeformRS-Par achieving a certified accuracy of 39\% against perturbed rotations in the set [-10 degree, 10 degree] on ImageNet.

show abstract

MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius

Cited by 24 publications

References 19 publications

Insta-RS: Instance-wise Randomized Smoothing for Improved Robustness and Accuracy

Insta-RS: Instance-wise Randomized Smoothing for Improved Robustness and Accuracy

Self-Progressing Robust Training

DeformRS: Certifying Input Deformations with Randomized Smoothing

Contact Info

Product

Resources

About