SqueezeNAS: Fast Neural Architecture Search for Faster Semantic Segmentation

Abstract

For real time applications utilizing Deep Neural Networks (DNNs), it is critical that the models achieve high-accuracy on the target task and low-latency inference on the target computing platform. While Neural Architecture Search (NAS) has been effectively used to develop low-latency networks for image classification, there has been relatively little effort to use NAS to optimize DNN architectures for other vision tasks. In this work, we present what we believe to be the first proxyless hardware-aware search targeted for dense semantic segmentation. With this approach, we advance the state-of-the-art accuracy for latency-optimized networks on the Cityscapes semantic segmentation dataset. Our latency-optimized small SqueezeNAS network achieves 68.02% validation class mIOU with less than 35 ms inference times on the NVIDIA Xavier. Our latency-optimized large SqueezeNAS network achieves 73.62% class mIOU with less than 100 ms inference times. We demonstrate that significant performance gains are possible by utilizing NAS to find networks optimized for both the specific task and inference hardware. We also present detailed analysis comparing our networks to recent state-of-the-art architectures.

Artificial neural network approach for multiphase segmentation of battery electrode nano-CT images

“…Other pseudo-3D CNN model uses adjacent slices as 3D input, but 2D convolution kernels as reported in 48,49 or 3D CNN model, which uses volume as inputs and 3D convolutions by Labonte et al 31 and an associated uncertainty metric 50 can be further investigated. There are also some emerging automatic techniques 51,52 searching optimal CNN architecture that could be potentially deployed in our current cases. Future direction might be to train a versatile network with a larger dataset for a specific collection of material.…”

SqueezeBERT: What can computer vision teach NLP about efficient neural networks?

“…The combination of all of these potential techniques opens up a broad search-space of neural architecture designs for NLP. This motivates the application of automated neural architecture search (NAS) approaches such as those described in (Shaw et al, 2019;Wu et al, 2019a) to further improve the design of neural networks for NLP.…”

ANNETTE: Accurate Neural Network Execution Time Estimation With Stacked Models

“…It offers potential for automated architecture search, especially when the demand for customized networks rises. FBnetV3 [25], SqueezeNAS [26] and Proxyless NAS [15] focus on low-latency network architecture search for mobile devices. They were developed to replace costly redesign DNNs for certain tasks on certain platforms.…”