Semisupervised Semantic Segmentation by Improving Prediction Confidence

Chen, Huaian; Jin, Yi; Jin, Guoqiang; Zhu, Changan; Chen, Enhong

doi:10.1109/tnnls.2021.3066850

Cited by 31 publications

(14 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semi-supervised learning (SSL) involves two typical paradigms: consistency regularization [3,48,73] and entropy minimization [5,6,27,49]. Consistency regularization forces the model to produce stable and consistent predictions on the same unlabeled data under various perturbations [71].…”

Section: Related Workmentioning

confidence: 99%

Iterative Loop Method Combining Active and Semi-Supervised Learning for Domain Adaptive Semantic Segmentation

Guan¹,

Yang²

2023

Preprint

View full text Add to dashboard Cite

Recently, self-training and active learning have been proposed to alleviate this problem. Self-training can improve model accuracy with massive unlabeled data, but some pseudo labels containing noise would be generated with limited or imbalanced training data. And there will be suboptimal models if human guidance is absent. Active learning can select more effective data to intervene, while the model accuracy can not be improved because the massive unlabeled data are not used. And the probability of querying sub-optimal samples will increase when the domain difference is too large, increasing annotation cost. This paper proposes an iterative loop learning method combining Self-Training and Active Learning (STAL) for domain adaptive semantic segmentation. The method first uses self-training to learn massive unlabeled data to improve model accuracy and provide more accurate selection models for active learning. Secondly, combined with the sample selection strategy of active learning, manual intervention is used to correct the self-training learning. Iterative loop to achieve the best performance with minimal label cost. Extensive experiments show that our method establishes stateof-the-art performance on tasks of GTAV → Cityscapes, SYNTHIA → Cityscapes, improving by 4.9% mIoU and 5.2% mIoU, compared to the previous best method, respectively. Code will be available.

show abstract

Section: Related Workmentioning

confidence: 99%

Iterative Loop Method Combining Active and Semi-Supervised Learning for Domain Adaptive Semantic Segmentation

Guan¹,

Yang²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…) 2 , and 3 = ∑ , 3 ( ( , )) 3 when the input is added with random noise for times ( is a constant integer). This smoothing strategy is intuitive but still rough for some complex CNN structures.…”

Section: Gradient-based Class Activation Mappingmentioning

confidence: 99%

“…Convolutional neural networks (CNNs) have provided a basis for numerous remarkable achievements in various computer vision tasks like, for example, image classification [9,8,27,13], object detection [19,43,1,12], and semantic segmentation [14,11,3]. Despite CNNs' extraordinary performance, they still lack a clear interpretation of the inner mechanism [10,42,21].…”

Section: Introductionmentioning

confidence: 99%

Cluster-CAM: Cluster-Weighted Visual Interpretation of CNNs' Decision in Image Classification

Feng¹,

Ji²,

Daković³

et al. 2023

Preprint

View full text Add to dashboard Cite

Despite the tremendous success of convolutional neural networks (CNNs) in computer vision, the mechanism of CNNs still lacks clear interpretation. Currently, class activation mapping (CAM), a famous visualization technique to interpret CNN's decision, has drawn increasing attention. Gradientbased CAMs are efficient while the performance is heavily affected by gradient vanishing and exploding. In contrast, gradient-free CAMs can avoid computing gradients to produce more understandable results. However, existing gradient-free CAMs are quite time-consuming because hundreds of forward interference per image are required. In this paper, we proposed Cluster-CAM, an effective and efficient gradient-free CNN interpretation algorithm. Cluster-CAM can significantly reduce the times of forward propagation by splitting the feature maps into clusters in an unsupervised manner. Furthermore, we propose an artful strategy to forge a cognition-base map and cognition-scissors from clustered feature maps. The final salience heatmap will be computed by merging the above cognition maps. Qualitative results conspicuously show that Cluster-CAM can produce heatmaps where the highlighted regions match the human's cognition more precisely than existing CAMs. The quantitative evaluation further demonstrates the superiority of Cluster-CAM in both effectiveness and efficiency.

show abstract

“…Existing supervised approaches rely on large-scale annotated data, which can be too costly to acquire in practice. To alleviate this problem, many attempts [1,4,9,15,21,33,43,48] have been made towards semi-supervised semantic segmentation, which learns a model with only a few labeled samples and numerous unlabeled ones. Under such a setting, how to adequately leverage the unlabeled data becomes critical.…”

Section: Iou Reliable Unreliablementioning

confidence: 99%

“…Semi-Supervised Learning has two typical paradigms: consistency regularization [3,15,33,36,42] and entropy minimization [4,16]. Recently, a more intuitive but effective framework: self-training [27], has become the mainstream.…”

Section: Related Workmentioning

confidence: 99%

Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Wang¹,

Wang²,

Shen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The crux of semi-supervised semantic segmentation is to assign adequate pseudo-labels to the pixels of unlabeled images. A common practice is to select the highly confident predictions as the pseudo ground-truth, but it leads to a problem that most pixels may be left unused due to their unreliability. We argue that every pixel matters to the model training, even its prediction is ambiguous. Intuitively, an unreliable prediction may get confused among the top classes (i.e., those with the highest probabilities), however, it should be confident about the pixel not belonging to the remaining classes. Hence, such a pixel can be convincingly treated as a negative sample to those most unlikely categories. Based on this insight, we develop an effective pipeline to make sufficient use of unlabeled data. Concretely, we separate reliable and unreliable pixels via the entropy of predictions, push each unreliable pixel to a category-wise queue that consists of negative samples, and manage to train the model with all candidate pixels. Considering the training evolution, where the prediction becomes more and more accurate, we adaptively adjust the threshold for the reliable-unreliable partition. Experimental results on various benchmarks and training settings demonstrate the superiority of our approach over the stateof-the-art alternatives. 1

show abstract

Semisupervised Semantic Segmentation by Improving Prediction Confidence

Cited by 31 publications

References 57 publications

Iterative Loop Method Combining Active and Semi-Supervised Learning for Domain Adaptive Semantic Segmentation

Iterative Loop Method Combining Active and Semi-Supervised Learning for Domain Adaptive Semantic Segmentation

Cluster-CAM: Cluster-Weighted Visual Interpretation of CNNs' Decision in Image Classification

Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Contact Info

Product

Resources

About