Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search

Xie, Guoyang; Wang, Jinbao; Yu, Guo; Feng, Zhiyong; Jin, Yaochu

doi:10.48550/arxiv.2103.00363

Cited by 3 publications

(2 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…propose Adversarially Robust Distillation (ARD), where they encourage student networks to mimic their teacher's output within an -ball of training samples. Furthermore, there are few NAS methods [32,58,63] that jointly optimises accuracy, latency and robustness. Compared to these methods, similar-sized RNAS-CL models achieve both higher clean and robust accuracy.…”

Section: Efficient and Robust Modelsmentioning

confidence: 99%

RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation

Nath¹,

Wang²,

Yang³

2023

Preprint

View full text Add to dashboard Cite

Deep Neural Networks are vulnerable to adversarial attacks. Neural Architecture Search (NAS), one of the driving tools of deep neural networks, demonstrates superior performance in prediction accuracy in various machine learning applications. However, it is unclear how it performs against adversarial attacks. Given the presence of a robust teacher, it would be interesting to investigate if NAS would produce robust neural architecture by inheriting robustness from the teacher. In this paper, we propose Robust Neural Architecture Search by Cross-Layer Knowledge Distillation (RNAS-CL), a novel NAS algorithm that improves the robustness of NAS by learning from a robust teacher through cross-layer knowledge distillation. Unlike previous knowledge distillation methods that encourage close student/teacher output only in the last layer, RNAS-CL automatically searches for the best teacher layer to supervise each student layer. Experimental result evidences the effectiveness of RNAS-CL and shows that RNAS-CL produces small and robust neural architecture.

show abstract

Section: Efficient and Robust Modelsmentioning

confidence: 99%

RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation

Nath¹,

Wang²,

Yang³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Among the existing studies, one type of approach is to use adversarial training during the search process to increase the robustness in the model. For example, some studies [8,20] adopted one-shot NAS technology. When training the supernet, adversarial training is used to enhance the robustness in the model.…”

Section: Related Workmentioning

confidence: 99%

Robustness Enhancement of Neural Networks via Architecture Search with Multi-Objective Evolutionary Optimization

et al. 2022

View full text Add to dashboard Cite

Along with the wide use of deep learning technology, its security issues have drawn much attention over the years. Adversarial examples expose the inherent vulnerability of deep learning models and make it a challenging task to improve their robustness. Model robustness is related not only to its parameters but also to its architecture. This paper proposes a novel robustness enhanced approach for neural networks based on a neural architecture search. First, we randomly sample multiple neural networks to construct the initial population. Second, we utilize the individual networks in the population to fit and update the surrogate models. Third, the population of neural networks is evolved through a multi-objective evolutionary algorithm, where the surrogate models accelerate the performance evaluation of networks. Finally, the second and third steps are performed alternately until a network architecture with high accuracy and robustness is achieved. Experimental results show that the proposed method outperforms some classical artificially designed neural networks and other architecture search algorithms in terms of robustness.

show abstract

Evolving Architectures With Gradient Misalignment Toward Low Adversarial Transferability

Operiano¹,

Pora

Iba³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Deep neural network image classifiers are known to be susceptible, not only to adversarial examples created for them, but also to those created for others. This phenomenon poses a potential security risk in various black-box systems that rely on image classifiers. One of the observations on networks that have transferability of adversarial examples between them is the similarity of their architectures. Networks with high architectural similarity tend to share high transferability as well. Thus, in this study, we address this problem from a novel perspective by investigating the contribution of network architecture to transferability. Specifically, we propose an architecture searching framework that employs neuroevolution to evolve network architectures and gradient misalignment loss to encourage networks to converge into dissimilar functions after training. Our findings indicate that the proposed framework successfully discovers architectures that reduce transferability from four standard networks, including ResNet and VGG, while maintaining good accuracy on unperturbed images. In addition, the evolved networks trained with gradient misalignment exhibit significantly lower transferability than a standard network trained with gradient misalignment, which indicates that network architecture plays an important role in reducing transferability. We demonstrate that designing or exploring proper network architectures is a promising approach to tackle the transferability issue and train adversarially robust image classifiers.

show abstract

Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search

Cited by 3 publications

References 40 publications

RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation

RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation

Robustness Enhancement of Neural Networks via Architecture Search with Multi-Objective Evolutionary Optimization

Evolving Architectures With Gradient Misalignment Toward Low Adversarial Transferability

Contact Info

Product

Resources

About