Smoothed Inference for Adversarially-Trained Models

Abstract

Deep neural networks are known to be vulnerable to inputs with maliciously constructed adversarial perturbations aimed at forcing misclassification. We study randomized smoothing as a way to both improve performance on unperturbed data as well as increase robustness to adversarial attacks. Moreover, we extend the method proposed by He et al. [16] by adding low-rank multivariate noise, which we then use as a base model for smoothing. The proposed method achieves 58.5% top-1 accuracy on CIFAR-10 under PGD attack and outperforms previous works by 4%. In addition, we consider a family of attacks, which were previously used for training purposes in the certified robustness scheme. We demonstrate that the proposed attacks are more effective than PGD against both smoothed and non-smoothed models. Since our method is based on sampling, it lends itself well for trading-off between the model inference complexity and its performance. A reference implementation of the proposed techniques is provided.

Enhanced Regularizers for Attributional Robustness

“…(Goodfellow, Shlens, and Szegedy 2015) introduced one-step Fast Gradient Sign Method (FGSM) attack which was followed by more effective iterative attacks such as (Kurakin, Goodfellow, and Bengio 2016), PGD attack (Madry et al 2018), Carlini Wagner attack (Carlini and Wagner 2017), Momentum Iterative attack (Dong et al 2018), Diverse Input Iterative attack (Xie et al 2019b), Jacobianbased saliency map approach (Papernot et al 2016), etc. A parallel line of work has also emerged on finding strategies to defend against stronger adversarial attacks such as Adversarial Training (Madry et al 2018), Adversarial Logit Pairing (Kannan, Kurakin, and Goodfellow 2018), Ensemble Adversarial Training (Tramèr et al 2018), Parsevals Network (Cisse et al 2017), Feature Denoising Training (Xie et al 2019a), Latent Adversarial Training (Kumari et al 2019, Jacobian Adversarial Regularizer (Chan et al 2020), Smoothed Inference (Nemcovsky et al 2019), etc. The recent work of (Zhang et al 2019) explored the trade-off between adversarial robustness and accuracy.…”

Enhanced Regularizers for Attributional Robustness

“…(Goodfellow, Shlens, and Szegedy 2015) introduced one-step Fast Gradient Sign Method (FGSM) attack which was followed by more effective iterative attacks such as (Kurakin, Goodfellow, and Bengio 2016), PGD attack (Madry et al 2018), Carlini Wagner attack (Carlini and Wagner 2017), Momentum Iterative attack (Dong et al 2018), Diverse Input Iterative attack (Xie et al 2019b), Jacobianbased saliency map approach (Papernot et al 2016), etc. A parallel line of work has also emerged on finding strategies to defend against stronger adversarial attacks such as Adversarial Training (Madry et al 2018), Adversarial Logit Pairing (Kannan, Kurakin, and Goodfellow 2018), Ensemble Adversarial Training (Tramèr et al 2018), Parsevals Network (Cisse et al 2017), Feature Denoising Training (Xie et al 2019a), Latent Adversarial Training (Kumari et al 2019, Jacobian Adversarial Regularizer (Chan et al 2020), Smoothed Inference (Nemcovsky et al 2019), etc. The recent work of (Zhang et al 2019) explored the trade-off between adversarial robustness and accuracy.…”