Efficient Semantic Segmentation Using Gradual Grouping

Abstract

Deep CNNs for semantic segmentation have high memory and run time requirements. Various approaches have been proposed to make CNNs efficient like grouped, shuffled, depth-wise separable convolutions. We study the effectiveness of these techniques on a real-time semantic segmentation architecture like ERFNet for improving runtime by over 5X. We apply these techniques to CNN layers partially or fully and evaluate the testing accuracies on Cityscapes dataset. We obtain accuracy vs parameters/FLOPs trade offs, giving accuracy scores for models that can run under specified runtime budgets.We further propose a novel training procedure which starts out with a dense convolution but gradually evolves towards a grouped convolution. We show that our proposed training method and efficient architecture design can improve accuracies by over 8% with depthwise separable convolutions applied on the encoder of ERFNet and attaching a light weight decoder. This results in a model which has a 5X improvement in FLOPs while only suffering a 4% degradation in accuracy with respect to ERFNet.

Real-Time Semantic Image Segmentation with Deep Learning for Autonomous Driving: A Survey

“…A structural unit of its architecture is presented in Figure 17. ESSGG [38] improves runtime of ERFNet by over a factor of 5X, by replacing its modules with more efficient ones, such as the aforementioned depthwise separable convolutions, grouped convolutions and channel shuffling. In this manner the inference time is reduced efficiently.…”