CAP: Context-Aware Pruning for Semantic Segmentation

Abstract

Network pruning for deep convolutional neural networks (CNNs) has recently achieved notable research progress on image-level classification. However, most existing pruning methods are not catered to or evaluated on semantic segmentation networks. In this paper, we advocate the importance of contextual information during channel pruning for semantic segmentation networks by presenting a novel Context-aware Pruning framework. Concretely, we formulate the embedded contextual information by leveraging the layer-wise channels interdependency via the Context-aware Guiding Module (CAGM) and introduce the Context-aware Guided Sparsification (CAGS) to adaptively identify the informative channels on the cumbersome model by inducing channel-wise sparsity on the scaling factors in batch normalization (BN) layers. The resulting pruned models require significantly lesser operations for inference while maintaining comparable performance to (at times outperforming) the original models. We evaluated our framework on widely-used benchmarks and showed its effectiveness on both large and lightweight models. On Cityscapes dataset, our framework reduces the number of parameters by 32%, 47%, 54%, and 63%, on PSPNet101, PSPNet50, ICNet, and SegNet, respectively, while preserving the performance.

Auto-Compressing Subset Pruning for Semantic Image Segmentation

“…To be able to evaluate ACOSP, we choose He et al (2021), You et al (2019), andBejnordi et al (2020) as baselines, which motivates our choice of datasets and architectures. Most notably, this excludes state-of-the-art semantic segmentation architectures, and we leave the study of these architectures to future work.…”

“…Pruning as a compression technique has been studied in great depth for image classification tasks (Blalock et al, 2020), but it has received less attention in the context of semantic segmentation. Latter, to the best of our knowledge, has only been discussed in a few prior works (He et al, 2021;You et al, 2019;Bejnordi et al, 2020).…”

“…Figure 1: Mean Intersection-over-Union (mIoU) as a function of the compression ratio for ACOSP and the baseline CAP (He et al, 2021). The data is obtained by training SegNet and PSPNet models on the CITYSCAPES dataset.…”

“…The table clearly shows the strength of ACOSP to keep a high performance despite a high compression ratio. † : Numbers are taken from(He et al, 2021). * : Numbers taken fromBejnordi et al (2020); Numbers were calculated on the CITYSCAPES validation set.…”

“…For CITYSCAPES, we report the mIoU on the test set if available and the validation set in parenthesis. ( † : numbers taken from(He et al, 2021))…”