Improved Adversarial Training via Learned Optimizer

Xiong, Yuanhao; Hsieh, Cho‐Jui

doi:10.1007/978-3-030-58598-3_6

Cited by 17 publications

(13 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A MORE DISCUSSIONS ON RELATED WORK Xiong & Hsieh (2020) propose a similar approach to ours that uses an RNN optimizer to learn attacks for adversarial training. The differences between our method and Xiong & Hsieh (2020) are three-fold.…”

Section: Discussionmentioning

confidence: 99%

“…Empirical results validate the superiority of our approach compared to the CNN-based generators. Meta-learning has also been used in black-box attacks (Du et al, 2020) and AT defenses (Xiong & Hsieh, 2020;Jiang et al, 2021). It is noteworthy that Xiong & Hsieh (2020) propose a similar approach to ours that uses an RNN optimizer to learn attacks for AT.…”

Section: Meta-learningmentioning

confidence: 99%

“…Meta-learning has also been used in black-box attacks (Du et al, 2020) and AT defenses (Xiong & Hsieh, 2020;Jiang et al, 2021). It is noteworthy that Xiong & Hsieh (2020) propose a similar approach to ours that uses an RNN optimizer to learn attacks for AT. Although the RNN optimizer is also adopted in our approach, we improve the attacking performance and generalization ability of the RNN optimizer with new techniques, as presented in Sec.…”

Section: Meta-learningmentioning

confidence: 99%

“…φ is calculated by truncated backpropagation. Besides the general framework of learning to optimize (Andrychowicz et al, 2016;Chen et al, 2020b;Xiong & Hsieh, 2020), there exist particular challenges in training the optimizer due to the optimization problem of adversarial attacks as illustrated below.…”

Section: Meta-attack With Recurrent Neural Networkmentioning

confidence: 99%

See 3 more Smart Citations

Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness

Yang¹,

Dong²,

Xiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

The vulnerability of deep neural networks to adversarial examples has motivated an increasing number of defense strategies for promoting model robustness. However, the progress is usually hampered by insufficient robustness evaluations. As the de facto standard to evaluate adversarial robustness, adversarial attacks typically solve an optimization problem of crafting adversarial examples with an iterative process. In this work, we propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the optimizer in adversarial attacks parameterized by a recurrent neural network, which is trained over a class of data samples and defenses to produce effective update directions during adversarial example generation. Furthermore, we develop a model-agnostic training algorithm to improve the generalization ability of the learned optimizer when attacking unseen defenses. Our approach can be flexibly incorporated with various attacks and consistently improves the performance with little extra computational cost. Extensive experiments demonstrate the effectiveness of the learned attacks by MAMA compared to the state-of-the-art attacks on different defenses, leading to a more reliable evaluation of adversarial robustness.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Meta-learningmentioning

confidence: 99%

Section: Meta-learningmentioning

confidence: 99%

Section: Meta-attack With Recurrent Neural Networkmentioning

confidence: 99%

See 2 more Smart Citations

Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness

Yang¹,

Dong²,

Xiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In [2], a modified RNN is introduced to improve scalability and generalizability of L2O, and [26] uses LSTM models and the attention mechanism for solving Bayesian swarm optimization. To better safeguard neural networks, [5], [27] improve the adversarial training sample generation by using L2O to solve a constrained maximization problem. In the context of wireless networks, [3], [4] apply L2O to optimize power allocation in multi-user interference channels for sum rate maximization.…”

Section: Related Workmentioning

confidence: 99%

Learning for Robust Combinatorial Optimization: Algorithm and Application

Shao¹,

Yang²,

Shen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Learning to optimize (L2O) has recently emerged as a promising approach to solving optimization problems by exploiting the strong prediction power of neural networks and offering lower runtime complexity than conventional solvers. While L2O has been applied to various problems, a crucial yet challenging class of problems -robust combinatorial optimization in the form of minimax optimization -have largely remained underexplored. In addition to the exponentially large decision space, a key challenge for robust combinatorial optimization lies in the inner optimization problem, which is typically non-convex and entangled with outer optimization. In this paper, we study robust combinatorial optimization and propose a novel learning-based optimizer, called LRCO (Learning for Robust Combinatorial Optimization), which quickly outputs a robust solution in the presence of uncertain context. LRCO leverages a pair of learningbased optimizers -one for the minimizer and the other for the maximizer -that use their respective objective functions as losses and can be trained without the need of labels for training problem instances. To evaluate the performance of LRCO, we perform simulations for the task offloading problem in vehicular edge computing. Our results highlight that LRCO can greatly reduce the worst-case cost and improve robustness, while having a very low runtime complexity.

show abstract

Scalable Learning to Optimize: A Learned Optimizer Can Train Big Models

Chen

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Improved Adversarial Training via Learned Optimizer

Cited by 17 publications

References 14 publications

Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness

Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness

Learning for Robust Combinatorial Optimization: Algorithm and Application

Scalable Learning to Optimize: A Learned Optimizer Can Train Big Models

Contact Info

Product

Resources

About