NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search

Zela, Arbër; Siems, Julien; Hutter, Frank

doi:10.48550/arxiv.2001.10422

Cited by 11 publications

(14 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The existing NAS methods could be summarized as these three categories: 1) Reinforcement learning based methods [21], [22]; 2) Evolution algorithm based methods [23], [23], [24]; and 3) Gradient based methods [25], [26], [27], [28], [29]. Recently, several NAS benchmarks for generic object classification task such as NAS-Bench-101 [30], NAS-Bench-201 [31], and NAS-Bench-1Shot1 [32] as well as evaluation manner [33] are proposed for fair performance comparison. In other side, in order to quickly adapt and discover excellent architectures in the unseen scenarios, some meta NAS based methods [34], [35], [36], [37], [38] are developed.…”

Section: Related Workmentioning

confidence: 99%

NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing

Wan

Qin

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

183

102

View full text Add to dashboard Cite

Face anti-spoofing (FAS) plays a vital role in securing face recognition systems. Existing methods heavily rely on the expert-designed networks, which may lead to a sub-optimal solution for FAS task. Here we propose the first FAS method based on neural architecture search (NAS), called NAS-FAS, to discover the well-suited task-aware networks. Unlike previous NAS works mainly focus on developing efficient search strategies in generic object classification, we pay more attention to study the search spaces for FAS task. The challenges of utilizing NAS for FAS are in two folds: the networks searched on 1) a specific acquisition condition might perform poorly in unseen conditions, and 2) particular spoofing attacks might generalize badly for unseen attacks. To overcome these two issues, we develop a novel search space consisting of central difference convolution and pooling operators. Moreover, an efficient static-dynamic representation is exploited for fully mining the FAS-aware spatio-temporal discrepancy. Besides, we propose Domain/Type-aware Meta-NAS, which leverages cross-domain/type knowledge for robust searching. Finally, in order to evaluate the NAS transferability for cross datasets and unknown attack types, we release a large-scale 3D mask dataset, namely CASIA-SURF 3DMask, for supporting the new 'cross-dataset cross-type' testing protocol. Experiments demonstrate that the proposed NAS-FAS achieves state-of-the-art performance on nine FAS benchmark datasets with four testing protocols.

show abstract

Section: Related Workmentioning

confidence: 99%

NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing

Wan

Qin

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

183

102

View full text Add to dashboard Cite

show abstract

“…In this series of experiments, we evaluate DEHB on a broad range of NAS benchmarks. We use a total of 13 tabular benchmarks from NAS-Bench-101 [Ying et al, 2019], NAS-Bench-1shot1 [Zela et al, 2020], NAS-Bench-201 [Dong and Yang, 2020] and NAS-HPO-Bench [Klein and Hutter, 2019]. For NAS-Bench-101, we show results on CifarC (a mixed data type encoding of the parameter space [Awad et al, 2020]) in Figure 10; BOHB and DEHB initially perform similarly as RS for this dataset, since there is only little correlation between runs with few epochs (low budgets) and many epochs (high budgets) in NAS-Bench-101.…”

Section: Nas Benchmarksmentioning

confidence: 99%

“…NAS-Bench-1shot1 was introduced by [Zela et al, 2020], as a benchmark derived from the large space of architectures offered by NAS-Bench-101. This benchmark allows the use of modern one-shot 8 NAS methods with weight sharing ( [Pham et al, 2018], [Liu et al, 2018]).…”

Section: Nas-bench-1shot1mentioning

confidence: 99%

DEHB: Evolutionary Hyberband for Scalable, Robust and Efficient Hyperparameter Optimization

Awad

Mallik

Hutter

2021

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Modern machine learning algorithms crucially rely on several design decisions to achieve strong performance, making the problem of Hyperparameter Optimization (HPO) more important than ever. Here, we combine the advantages of the popular bandit-based HPO method Hyperband (HB) and the evolutionary search approach of Differential Evolution (DE) to yield a new HPO method which we call DEHB. Comprehensive results on a very broad range of HPO problems, as well as a wide range of tabular benchmarks from neural architecture search, demonstrate that DEHB achieves strong performance far more robustly than all previous HPO methods we are aware of, especially for high-dimensional problems with discrete input dimensions. For example, DEHB is up to 1000x faster than random search. It is also efficient in computational time, conceptually simple and easy to implement, positioning it well to become a new default HPO method.

show abstract

“…There are several important directions that we intend to explore in the future, including (i) simplify the adjacency matrix P to capture the dependency and mutual interaction between synaptic connections, e.g., approximate gradients using local information (Jaderberg et al, 2017), (ii) extend the proposed framework to NAS benchmarks (Ying et al, 2019;Dong & Yang, 2020;Dong et al, 2021;Zela et al, 2020; to select the best subnetwork, and (iii) design an efficient algorithm to directly optimize NN architectures based on β eff .…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics

Jiang¹,

Pedapati²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Efficient model selection for identifying a suitable pre-trained neural network to a downstream task is a fundamental yet challenging task in deep learning. Current practice requires expensive computational costs in model training for performance prediction. In this paper, we propose a novel framework for neural network selection by analyzing the governing dynamics over synaptic connections (edges) during training. Our framework is built on the fact that back-propagation during neural network training is equivalent to the dynamical evolution of synaptic connections. Therefore, a converged neural network is associated with an equilibrium state of a networked system composed of those edges. To this end, we construct a network mapping φ, converting a neural network G A to a directed line graph G B that is defined on those edges in G A . Next, we derive a neural capacitance metric β eff as a predictive measure universally capturing the generalization capability of G A on the downstream task using only a handful of early training results. We carried out extensive experiments using 17 popular pre-trained ImageNet models and five benchmark datasets, including CIFAR10, CIFAR100, SVHN, Fashion MNIST and Birds, to evaluate the fine-tuning performance of our framework. Our neural capacitance metric is shown to be a powerful indicator for model selection based only on early training results and is more efficient than state-of-the-art methods.Preprint.

show abstract

NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search

Cited by 11 publications

References 3 publications

NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing

NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing

DEHB: Evolutionary Hyberband for Scalable, Robust and Efficient Hyperparameter Optimization

Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics

Contact Info

Product

Resources

About