Adversarially Robust Learning via Entropic Regularization

Jagatap, Gauri; Joshi, Ameya; Chowdhury, Animesh Basak; Garg, Siddharth; Hegde, Chinmay

doi:10.3389/frai.2021.780843

Cited by 8 publications

(4 citation statements)

References 23 publications

(52 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, they do not show EnSGD to be effective for nonsmooth loss functions. In addition, though several studies [35], [36] apply Langevin dynamics to generate attacks, there are few studies that apply it to optimize parameters in AT. Thus, the effectiveness for AT is still unclear due to nonsmoothness.…”

Section: Entropysgdmentioning

confidence: 99%

Relationship Between Nonsmoothness in Adversarial Training, Constraints of Attacks, and Flatness in the Input Space

Kanai

Yamada

Takahashi³

et al. 2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Adversarial training (AT) is a promising method to improve the robustness against adversarial attacks. However, its performance is not still satisfactory in practice compared with standard training. To reveal the cause of the difficulty of AT, we analyze the smoothness of the loss function in AT, which determines the training performance. We reveal that nonsmoothness is caused by the constraint of adversarial attacks and depends on the type of constraint. Specifically, the L∞ constraint can cause nonsmoothness more than the L2 constraint. In addition, we found an interesting property for AT: the flatter loss surface in the input space tends to have the less smooth adversarial loss surface in the parameter space. To confirm that the nonsmoothness causes the poor performance of AT, we theoretically and experimentally show that smooth adversarial loss by EntropySGD (EnSGD) improves the performance of AT.

show abstract

Section: Entropysgdmentioning

confidence: 99%

Relationship Between Nonsmoothness in Adversarial Training, Constraints of Attacks, and Flatness in the Input Space

Kanai

Yamada

Takahashi³

et al. 2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…This aligns with [28], where the authors demonstrated that the local Lipschitz constant can be used to explicitly quantify the robustness of machine learning models. Many empirical defensive approaches, such as hessian/curvature-based regularization [48], gradient magnitude penalty [73], smoothening with random noise [44], or entropy regularization [33], have echoed the flatness performance of a robust model. However, all the above approaches require significant computational or memory resources, and many of them, such as hessian/curvature-based solutions, may suffer from standard accuracy decreases [26].…”

Section: Motivationmentioning

confidence: 99%

Randomized Adversarial Training via Taylor Expansion

Jin¹,

Xiao²,

Wu³

et al. 2023

Preprint

View full text Add to dashboard Cite

In recent years, there has been an explosion of research into developing more robust deep neural networks against adversarial examples. Adversarial training appears as one of the most successful methods. To deal with both the robustness against adversarial examples and the accuracy over clean examples, many works develop enhanced adversarial training methods to achieve various trade-offs between them [19,38,82]. Leveraging over the studies [8,32] that smoothed update on weights during training may help find flat minima and improve generalization, we suggest reconciling the robustness-accuracy trade-off from another perspective, i.e., by adding random noise into deterministic weights. The randomized weights enable our design of a novel adversarial training method via Taylor expansion of a small Gaussian noise, and we show that the new adversarial training method can flatten loss landscape and find flat minima. With PGD, CW, and Auto Attacks, an extensive set of experiments demonstrate that our method enhances the state-of-the-art adversarial training methods, boosting both robustness and clean accuracy. The code is available at https://github.com/Alexkael/Randomized-Adversarial-Training.

show abstract

“…We adopt adversarial re-training [11], [12] to train a model that is exposed to a wide variation of localities, i.e., subgraph structures around key-gates. The motivation for adversarial retraining is as follows: we want to create adversarial subgraph embeddings where the attack model M *…”

Section: B Adversarially Trained Attacker's Model M *mentioning

confidence: 99%

ALMOST: Adversarial Learning to Mitigate Oracle-less ML Attacks via Synthesis Tuning

Chowdhury¹,

Alrahis²,

Collini³

et al. 2023

Preprint

View full text Add to dashboard Cite

Oracle-less machine learning (ML) attacks have broken various logic locking schemes. Regular synthesis, which is tailored for area-power-delay optimization, yields netlists where key-gate localities are vulnerable to learning. Thus, we call for security-aware logic synthesis. We propose ALMOST, a framework for adversarial learning to mitigate oracle-less ML attacks via synthesis tuning. ALMOST uses a simulated-annealingbased synthesis recipe generator, employing adversarially trained models that can predict state-of-the-art attacks' accuracies over wide ranges of recipes and key-gate localities. Experiments on ISCAS benchmarks confirm the attacks' accuracies drops to around 50% for ALMOST-synthesized circuits, all while not undermining design optimization.

show abstract

Adversarially Robust Learning via Entropic Regularization

Cited by 8 publications

References 23 publications

Relationship Between Nonsmoothness in Adversarial Training, Constraints of Attacks, and Flatness in the Input Space

Relationship Between Nonsmoothness in Adversarial Training, Constraints of Attacks, and Flatness in the Input Space

Randomized Adversarial Training via Taylor Expansion

ALMOST: Adversarial Learning to Mitigate Oracle-less ML Attacks via Synthesis Tuning

Contact Info

Product

Resources

About