Single-Path NAS: Designing Hardware-Efficient ConvNets in Less Than 4 Hours

Stamoulis, Dimitrios; Ding, Ruixue; Wang, Di; Lymberopoulos, Dimitrios; Priyantha, Bodhi; Liu, Jie; Marculescu, Diana

doi:10.1007/978-3-030-46147-8_29

Cited by 138 publications

(97 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hardware-aware NAS Hardware-aware NAS methods can be roughly categorized into two branches, namely regularization-based and hard constraint methods. Most regularization-based methods [22][23] view latency as differentiable regularization loss. Many one-shot NAS methods [10][11] alterly use hard constraint paradigm, which directly discard architectures that exceeds the constraints in the search phase.…”

Section: Related Workmentioning

confidence: 99%

Hardware-Aware Low-Light Image Enhancement via One-Shot Neural Architecture Search with Shrinkage Sampling

Yao

Liu

Zhang

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

Low-light image enhancement has traditionally been tackled by training a heuristically designed neural network architecture. Despite the success of these approaches, the heuristic design pattern inherently not only hinders further optimization of network architectures, but also limits the factors that the designer can take into consideration. As a result, these methods are difficult to achieve a balance between enhancing performance and hardware related performance. In this paper, we equip a basic enhancing algorithm with a neural architecture search technique. This technique helps to automatically search an optimal hardware-aware architecture while also increases neglectable computation burden. In this work, we propose a shrinkage sampling strategy to drastically decrease the computation cost of neural architecture search while improving the quality of search. Extensive experiments on various benchmarks demonstrate that our algorithm achieves state-of-the-art performance with higher speed.

show abstract

Section: Related Workmentioning

confidence: 99%

Hardware-Aware Low-Light Image Enhancement via One-Shot Neural Architecture Search with Shrinkage Sampling

Yao

Liu

Zhang

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

show abstract

“…For the former one, there exist sandwich rule (Yu and Huang 2019), uniform sampling (Guo et al 2019), fairness-aware sampling methods (Chu et al 2019) and so on. For the latter one, there exist greedy (Yu and Huang 2019), evolutionary (Guo et al 2019), differentiable searching methods (Stamoulis et al 2019). These two operations are applied to make the candidate networks predictable and search the best one efficiently.…”

Section: Neural Architecture Searchmentioning

confidence: 99%

Neural Inheritance Relation Guided One-Shot Layer Assignment Search

Meng

Chen

Xie

et al. 2020

AAAI

View full text Add to dashboard Cite

Layer assignment is seldom picked out as an independent research topic in neural architecture search. In this paper, for the first time, we systematically investigate the impact of different layer assignments to the network performance by building an architecture dataset of layer assignment on CIFAR-100. Through analyzing this dataset, we discover a neural inheritance relation among the networks with different layer assignments, that is, the optimal layer assignments for deeper networks always inherit from those for shallow networks. Inspired by this neural inheritance relation, we propose an efficient one-shot layer assignment search approach via inherited sampling. Specifically, the optimal layer assignment searched in the shallow network can be provided as a strong sampling priori to train and search the deeper ones in supernet, which extremely reduces the network search space. Comprehensive experiments carried out on CIFAR-100 illustrate the efficiency of our proposed method. Our search results are strongly consistent with the optimal ones directly selected from the architecture dataset. To further confirm the generalization of our proposed method, we also conduct experiments on Tiny-ImageNet and ImageNet. Our searched results are remarkably superior to the handcrafted ones under the unchanged computational budgets. The neural inheritance relation discovered in this paper can provide insights to the universal neural architecture search.

show abstract

“…Only a few of them have recently introduced the hardware constraints in the optimization problem, for instance, considering the hardware resources (e.g., #FLOPs, memory requirements, etc.) available for performing the DNN inference [9][10] [14] [30]. To the best of our knowledge, none of them include in the design space the possibility of employing capsule layers and dynamic routing, which are inevitable for automatically designing the CapsNets.…”

Section: Research Problem and Associated Challengesmentioning

confidence: 99%