Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation

Abstract

Recent semantic segmentation methods exploit encoderdecoder architectures to produce the desired pixel-wise segmentation prediction. The last layer of the decoders is typically a bilinear upsampling procedure to recover the final pixel-wise prediction. We empirically show that this oversimple and data-independent bilinear upsampling may lead to sub-optimal results.In this work, we propose a data-dependent upsampling (DUpsampling) to replace bilinear, which takes advantages of the redundancy in the label space of semantic segmentation and is able to recover the pixel-wise prediction from low-resolution outputs of CNNs. The main advantage of the new upsampling layer lies in that with a relatively lowerresolution feature map such as 1 16 or 1 32 of the input size, we can achieve even better segmentation accuracy, significantly reducing computation complexity. This is made possible by 1) the new upsampling layer's much improved reconstruction capability; and more importantly 2) the DUpsampling based decoder's flexibility in leveraging almost arbitrary combinations of the CNN encoders' features. Experiments demonstrate that our proposed decoder outperforms the state-of-the-art decoder, with only ∼20% of computation. Finally, without any post-processing, the framework equipped with our proposed decoder achieves new state-of-the-art performance on two datasets: 88.1% mIOU on PASCAL VOC with 30% computation of the previously best model; and 52.5% mIOU on PASCAL Context.

Adaptive Context Network for Scene Parsing

“…ACNet obtains a Mean IoU of 54.1%, which surpasses previous published methods. Among the approaches, the recent methods [21,28] use more powerful network(e.g. ResNet-152 and Xception-71) as encoder network and fuse high-and low-level feature in decoder network, our method outperforms them by a relatively large margin.…”

“…RefineNet [22] 47.3 MSCI [21] 50.3 Xception-71 Tian et al [28] 52.5 ploying a deeper backbone ResNet101, ACNet achieves a new state-of-the-art performance of 45.90%/81.96%, which outperforms the previous state-of-the-art methods. In addition, we also fine tune our best model of ACNet-101 with trainval data, and submit our test results on the test set.…”

Robust Method for Semantic Segmentation of Whole-Slide Blood Cell Microscopic Images

“…It measures the percent of boundary match between ground truth boundary and predicted boundary of an object [28]. It is a combined ratio of twice of precision and recall product over sum of recall and precision: 8 Computational and Mathematical Methods in Medicine BF score � 2 * (Precision * Recall) Recall + Precision .…”

“…But, none of the previous techniques used the wholeslide segmentation of all three types of blood cells simultaneously. Semantic segmentation became popular for dense pixel prediction of objects in images using fully convolutional networks (FCNs) [8,9]. ese networks are penetrated deeply for the detection, classification, and prediction of the pixel base region of interest (ROI).…”

SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection

“…In this subsection, we evaluate the performance of our proposed SNE-RoadSeg-152 (abbreviated as SNE-RoadSeg) both qualitatively and quantitatively. Ex- , U-Net [26], SegNet [3], GSCNN [30], DUpsampling [32] and DenseASPP [37] with and without our SNE embedded, where RGB, Depth, and SNE-Depth (Ours).…”