Addressing Neural Network Robustness with Mixup and Targeted Labeling Adversarial Training

Laugros, Alfred; Caplier, Alice; Ospici, Matthieu

doi:10.48550/arxiv.2008.08384

Cited by 2 publications

(2 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To defend against adaptive attacks, several mixup-based adversarial training techniques have been proposed (Lamb et al, 2019;Laugros et al, 2020). Interpolated adversarial training (IAT) (Lamb et al, 2019) trains on interpolations of adversarial data along with interpolations of natural data.…”

Section: Mixup For Robustnessmentioning

confidence: 99%

“…Interpolated adversarial training (IAT) (Lamb et al, 2019) trains on interpolations of adversarial data along with interpolations of natural data. Mixup with targeted labeling adversarial training (M-TLAT) (Laugros et al, 2020) combines vanilla mixup with targeted labeling to enhance AT. However, these methods behave linearly when employing vanilla mixup in AT, and such linear behaviors will damage the adversarial robustness (see Section 3.1 for details).…”

Section: Mixup For Robustnessmentioning

confidence: 99%

See 1 more Smart Citation

Guided Interpolation for Adversarial Training

Chen,

Zhang,

et al. 2021

Preprint

View full text Add to dashboard Cite

To enhance adversarial robustness, adversarial training learns deep neural networks on the adversarial variants generated by their natural data. However, as the training progresses, the training data becomes less and less attackable, undermining the robustness enhancement. A straightforward remedy is to incorporate more training data, but sometimes incurring an unaffordable cost. In this paper, to mitigate this issue, we propose the guided interpolation framework (GIF): in each epoch, the GIF employs the previous epoch's meta information to guide the data's interpolation. Compared with the vanilla mixup, the GIF can provide a higher ratio of attackable data, which is beneficial to the robustness enhancement; it meanwhile mitigates the model's linear behavior between classes, where the linear behavior is favorable to standard training for generalization but not to adversarial training for robustness. As a result, the GIF encourages the model to predict invariantly in the cluster of each class. Experiments demonstrate that the GIF can indeed enhance adversarial robustness on various adversarial training methods and various datasets.Recent studies on AT suggest the unequal treatment of data (Ding et al., 2020;Wang et al., 2019;Zhang et al., 2021a). In particular, Zhang et al. (2021a) divided the training data into two categories-attackable data and guarded data, in which attackable/guarded data are close to/far away from the class boundary that can/cannot be attacked. To enhance adversarial robustness, attackable data are particularly useful in learning the decision boundary Zhang et al., 2021a).However, as the AT progresses, the ratio of attackable data decreases significantly, which jeopardizes the enhancement of adversarial robustness. In Figure 1, we plot the ratio of attackable data in the left panel and the robust accuracy in the right panel. The red lines show the training dynamics of a typical AT method . As the training progresses, more and more training data become guarded; thus, the ratio of attackable data decreases. After Epoch 30 (with a reduced learning rate), this ratio drops rapidly, while the robust accuracy ceases to rise but begins to drop. This strong correlation between this ratio and the robustness urges us to introduce more attackable data for AT.A straightforward remedy for the shortage of attackable data is to incorporate more training data . Hendrycks et al. (2019) showed that AT for learning a robust model requires substantially more training data than standard training (ST). Nevertheless, gathering additional data especially with high-quality * Equal contributions.

show abstract

Section: Mixup For Robustnessmentioning

confidence: 99%

Section: Mixup For Robustnessmentioning

confidence: 99%

Guided Interpolation for Adversarial Training

Chen,

Zhang,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities

Tocchetti,

Corti,

Balayn

et al. 2024

ACM Comput. Surv.

View full text Add to dashboard Cite

Despite the impressive performance of Artificial Intelligence (AI) systems, their robustness remains elusive and constitutes a key issue that impedes large-scale adoption. Besides, robustness is interpreted differently across domains and contexts of AI. In this work, we systematically survey recent progress to provide a reconciled terminology of concepts around AI robustness. We introduce three taxonomies to organize and describe the literature both from a fundamental and applied point of view: 1) methods and approaches that address robustness in different phases of the machine learning pipeline; 2) methods improving robustness in specific model architectures, tasks, and systems; and in addition, 3) methodologies and insights around evaluating the robustness of AI systems, particularly the trade-offs with other trustworthiness properties. Finally, we identify and discuss research gaps and opportunities and give an outlook on the field. We highlight the central role of humans in evaluating and enhancing AI robustness, considering the necessary knowledge they can provide, and discuss the need for better understanding practices and developing supportive tools in the future.

show abstract

Addressing Neural Network Robustness with Mixup and Targeted Labeling Adversarial Training

Cited by 2 publications

References 32 publications

Guided Interpolation for Adversarial Training

Guided Interpolation for Adversarial Training

A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities

Contact Info

Product

Resources

About