SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection

Abstract

The state-of-the-art object detection method is complicated with various modules such as backbone, feature fusion neck, RPN, and RCNN head, where each module may have different designs and structures. How to leverage the computational cost and accuracy trade-off for the structural combination as well as the modular selection of multiple modules? Neural architecture search (NAS) has shown great potential in finding an optimal solution. Existing NAS works for object detection only focus on searching better design of a single module such as backbone or feature fusion neck, while neglecting the balance of the whole system. In this paper, we present a two-stage coarse-to-fine searching strategy named Structural-to-Modular NAS (SM-NAS) for searching a GPU-friendly design of both an efficient combination of modules and better modular-level architecture for object detection. Specifically, Structural-level searching stage first aims to find an efficient combination of different modules; Modular-level searching stage then evolves each specific module and pushes the Pareto front forward to a faster task-specific network. We consider a multi-objective search where the search space covers many popular designs of detection methods. We directly search a detection backbone without pre-trained models or any proxy task by exploring a fast training from scratch strategy. The resulting architectures dominate state-of-the-art object detection systems in both inference time and accuracy and demonstrate the effectiveness on multiple detection datasets, e.g. halving the inference time with additional 1% mAP improvement compared to FPN and reaching 46% mAP with the similar inference time of MaskRCNN.

Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation

“…our searched detector based on R101 reaches 23.3 FPS and 43.9 AP, outperforming RepPoint-R101's 13.7 FPS and 41.0 AP by a large margin. Furthermore, our method (R101-based) surpasses most mainstream detection NAS methods (e.g., SM-NAS [42] and NAS-FPN [13]) and reaches comparable performance with the SOTA EfficientDet (D2) [37], while requiring much less search cost and no extra post-search training epochs.…”

“…changes, which calls for a way of obtaining sufficient powerful teachers with low cost. The mainstream NAS approach [13,42,37] using a proxy task (e.g., training with fewer epochs) to train the teacher does not guarantee the quality of teacher's supervision. On the other hand, training every teacher detector from scratch is too costly.…”

“…Finding the optimal tradeoff between model performance and complexity has always been a core problem for the community. The mainstream approaches aiming at addressing this issue are: Neural Architecture Search (NAS) [13,8,42,17] is proposed to automatically search for promising model architectures; pruning [18,20,28] removes redundant parameters from a model while maintaining its performance; and Knowledge Distillation (KD) [16,5,19,40,9] Figure 1: Limitation of NAS, pruning and KD. NAS is parameter-agnostic: The model search and training processes are decoupled, the searched architecture is retrained from scratch; Pruning is structure-agnostic: the pre-trained model has a fixed architecture; KD transfers knowledge between a fixed student-teacher pair while neglecting the structural dependence between the student and the teacher.…”

G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation

“…[30,6,33] propose to directly predict location of objects rather than based on anchor priors, which opens a new era for object detection. Recent works also perform NAS on detection tasks, which searches for novel detectors automatically without human intervention [13,32,10,34,35].…”

TransNAS-Bench-101: Improving transferability and Generalizability of Cross-Task Neural Architecture Search

“…[27] seeks to bridge the gap between sample-based and one-shot approaches. Besides searching for an optimal cell design in earlier works [19], many recent works also investigate the macro skeleton search of a network [33,31].…”