Context Encoding for Semantic Segmentation

Zhang, Hang; Dana, Kristin; Shi, Jianping; Zhang, Zhongyue; Wang, Xiaogang; Tyagi, Ambrish; Agrawal, Amit

doi:10.1109/cvpr.2018.00747

Cited by 1,356 publications

(955 citation statements)

References 62 publications

(136 reference statements)

Supporting

Mentioning

954

Contrasting

Unclassified

Order By: Relevance

“…Following previous works [16,23], we choose the most frequent 59 classes plus one background class (i.e., 60 classes in total) in our experiments. There is not a test server available and therefore we follow previous works [16,34,3,21,37] to report our result on val set.…”

Section: Methodsmentioning

confidence: 99%

“…Finally, we compare our proposed decoder with the vanilla bilinear decoder on the Cityscapes val set. Follow-Method mIOU (%) PSPNet [36] 85.4 DeepLabv3 [4] 85.7 EncNet [34] 85.9 DFN [32] 86.2 IDW-CNN [27] 86.3 CASIA IVA SDN [9] 86.6 DIS [22] 86.8 DeepLabv3+ [5] (Xception-65) 87.8 Our proposed (Xception-65) 88.1 ing [5], Xception-71 is used as our backbone and the number of iterations is increased to 90k with a initial learning rate being 0.01. As shown in Table 4, under the same training and testing settings, our proposed decoder achieves a comparable performance with the vanilla one while using much less computation.…”

Section: Comparison With the Vanilla Bilinear Decodermentioning

confidence: 99%

“…Method mIOU (%) FCN-8s [21] 37.8 CRF-RNN [37] 39.3 HO CRF [1] 41.3 Piecewise [17] 43.3 VeryDeep [30] 44.5 DeepLabv2 [3] 45.7 RefineNet [16] 47.3 EncNet [34] 51.7 Our proposed (Xception-65) 51.4 Our proposed (Xception-71) 52.5 Table 6: State-of-the-art methods on PASCAL Context val set.…”

Section: Comparison With the Vanilla Bilinear Decodermentioning

confidence: 99%

“…In this experiment, following previous works [31,36,34] without COCO pre-training, we train our model on SBD [10] and then fine-tune it on official trainval set. We use the same training protocol as described in the main paper.…”

Section: Pascal Voc Without Coco Pre-trainingmentioning

confidence: 99%

“…Method mIOU (%) DPN [20] 74.1 Piecewise [17] 75.3 ResNet-38 [31] 82.5 PSPNet [36] 82.6 DFN [32] 82.7 EncNet [34] 82.9 Our proposed (Xception-65) 85.3 Table 7: State-of-the-art methods on PASCAL VOC test set without COCO pre-training.…”

Section: Pascal Voc Without Coco Pre-trainingmentioning

confidence: 99%

See 4 more Smart Citations

Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation

Tian

Shen

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

251

146

View full text Add to dashboard Cite

Recent semantic segmentation methods exploit encoderdecoder architectures to produce the desired pixel-wise segmentation prediction. The last layer of the decoders is typically a bilinear upsampling procedure to recover the final pixel-wise prediction. We empirically show that this oversimple and data-independent bilinear upsampling may lead to sub-optimal results.In this work, we propose a data-dependent upsampling (DUpsampling) to replace bilinear, which takes advantages of the redundancy in the label space of semantic segmentation and is able to recover the pixel-wise prediction from low-resolution outputs of CNNs. The main advantage of the new upsampling layer lies in that with a relatively lowerresolution feature map such as 1 16 or 1 32 of the input size, we can achieve even better segmentation accuracy, significantly reducing computation complexity. This is made possible by 1) the new upsampling layer's much improved reconstruction capability; and more importantly 2) the DUpsampling based decoder's flexibility in leveraging almost arbitrary combinations of the CNN encoders' features. Experiments demonstrate that our proposed decoder outperforms the state-of-the-art decoder, with only ∼20% of computation. Finally, without any post-processing, the framework equipped with our proposed decoder achieves new state-of-the-art performance on two datasets: 88.1% mIOU on PASCAL VOC with 30% computation of the previously best model; and 52.5% mIOU on PASCAL Context.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Comparison With the Vanilla Bilinear Decodermentioning

confidence: 99%

Section: Comparison With the Vanilla Bilinear Decodermentioning

confidence: 99%

Section: Pascal Voc Without Coco Pre-trainingmentioning

confidence: 99%

Section: Pascal Voc Without Coco Pre-trainingmentioning

confidence: 99%

See 3 more Smart Citations

Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation

Tian

Shen

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

251

146

View full text Add to dashboard Cite

show abstract

Temporal Trends in Cervical Spine Curvature of South Korean Adults Assessed by Deep Learning System Segmentation, 2006-2018

2020

View full text Add to dashboard Cite

show abstract

A robust interclass and intraclass loss function for deep learning based tongue segmentation

Cai

Wang

Liu

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

The fast and robust segmentation of tongue images is a prerequisite to achieve automatic tongue diagnosis in traditional Chinese medicine. In order to assist tongue diagnosis in real-life scenarios, an ideal tongue segmentation method would need to obtain the entire tongue body as well as its precise contours. However, the similar appearance among the tongue body, the coating, and the lips hinders the performance for most unsupervised learning methods that primarily utilize low-level visual features. On the other hand, although the supervised deep convolutional neural networks (DCNNs) that typically depend on the widely used crossentropy loss can achieve better accuracy, they are very prone to segment image to multiple trivially areas. To address both of the above issues, we make an attempt to boost the segmentation performance of DCNNs with a novel auxiliary loss function that seeks to exploit large margin learning for end-to-end tongue segmentation models. Specifically, we first propose a loss function that involves interclass and intraclass costs to directly measure the distance among pixels that belong to different connected regions. Then, we explore the potential ability of this loss function as a regularization for different segmentation networks such as those with attention modules and the deeply supervised network. Finally, a theoretical analysis for this learning scheme is presented. Through experiments on challenging datasets, we show that the proposed approach can be easily integrated into state-of-the-art networks to boost their performance at the tongue segmentation task without bells and whistles.

show abstract

Context Encoding for Semantic Segmentation

Cited by 1,356 publications

References 62 publications

Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation

Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation

Temporal Trends in Cervical Spine Curvature of South Korean Adults Assessed by Deep Learning System Segmentation, 2006-2018

A robust interclass and intraclass loss function for deep learning based tongue segmentation

Contact Info

Product

Resources

About