Generating Accurate Pseudo-Labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations

Lokhande, Vishnu Suresh; Tasneeyapant, Songwong; Venkatesh, Abhay; Ravi, Sathya N.; Singh, Vikas

doi:10.1109/cvpr42600.2020.01145

Cited by 24 publications

(14 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is a limitation of pseudo-label methods, and leads to considerable errors in the pseudo labels generated from a model with poor performance. Because these false predictions are treated as ground-truth labels for training, when the error rate in certain categories is too high, false pseudo-labels dominate during subsequent training iterations and contribute to a side effect in the model performance with respect to these categories, which provides overconfident predictions (Lokhande et al, 2020).…”

Section: Results and Analysismentioning

confidence: 99%

Weakly Supervised Pseudo-Label Assisted Learning for Als Point Cloud Semantic Segmentation

Wang

Yao

2021

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. Competitive point cloud semantic segmentation results usually rely on a large amount of labeled data. However, data annotation is a time-consuming and labor-intensive task, particularly for three-dimensional point cloud data. Thus, obtaining accurate results with limited ground truth as training data is considerably important. As a simple and effective method, pseudo labels can use information from unlabeled data for training neural networks. In this study, we propose a pseudo-label-assisted point cloud segmentation method with very few sparsely sampled labels that are normally randomly selected for each class. An adaptive thresholding strategy was proposed to generate a pseudo-label based on the prediction probability. Pseudo-label learning is an iterative process, and pseudo labels were updated solely on ground-truth weak labels as the model converged to improve the training efficiency. Experiments using the ISPRS 3D sematic labeling benchmark dataset indicated that our proposed method achieved an equally competitive result compared to that using a full supervision scheme with only up to 2‰ of labeled points from the original training set, with an overall accuracy of 83.7% and an average F1 score of 70.2%.

show abstract

Section: Results and Analysismentioning

confidence: 99%

Weakly Supervised Pseudo-Label Assisted Learning for Als Point Cloud Semantic Segmentation

Wang

Yao

2021

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…The consistency loss is based on the idea that

, where

is an unlabeled image, and “

” refers to performing horizontal mirroring. Lokhande et al [ 22 ] used self-training for deep image classification. In this case, the original activation functions of

, a CNN for image classification, must be changed to Hermite polynomials.…”

Section: Related Workmentioning

confidence: 99%

“…In this case, the original activation functions of

, a CNN for image classification, must be changed to Hermite polynomials. Note that these two examples of self-training involve modifications either in the architecture of

[ 22 ] or in its training framework [ 21 ]. However, we aim at using a given

together with its training framework as a black box, so performing SSL only at the data level.…”

Section: Related Workmentioning

confidence: 99%

Co-Training for Deep Object Detection: Comparing Single-Modal and Multi-Modal Approaches

Gómez

Villalonga

López

2021

Sensors

View full text Add to dashboard Cite

Top-performing computer vision models are powered by convolutional neural networks (CNNs). Training an accurate CNN highly depends on both the raw sensor data and their associated ground truth (GT). Collecting such GT is usually done through human labeling, which is time-consuming and does not scale as we wish. This data-labeling bottleneck may be intensified due to domain shifts among image sensors, which could force per-sensor data labeling. In this paper, we focus on the use of co-training, a semi-supervised learning (SSL) method, for obtaining self-labeled object bounding boxes (BBs), i.e., the GT to train deep object detectors. In particular, we assess the goodness of multi-modal co-training by relying on two different views of an image, namely, appearance (RGB) and estimated depth (D). Moreover, we compare appearance-based single-modal co-training with multi-modal. Our results suggest that in a standard SSL setting (no domain shift, a few human-labeled data) and under virtual-to-real domain shift (many virtual-world labeled data, no human-labeled data) multi-modal co-training outperforms single-modal. In the latter case, by performing GAN-based domain translation both co-training modalities are on par, at least when using an off-the-shelf depth estimation model not specifically trained on the translated images.

show abstract

“…The consistency loss is based on the idea that φ(I u ) ∼ φ(I u ) , where I u is an unlabeled image, and " " refers to performing horizontal mirroring. Lokhande et al [15] used self-training for deep image classification. In this case, the original activation functions of φ, a CNN for image classification, must be changed to Hermite polynomials.…”

Section: Related Workmentioning

confidence: 99%

“…In this case, the original activation functions of φ, a CNN for image classification, must be changed to Hermite polynomials. Note that these two examples of self-training involve modifications either in the architecture of φ [15] or in its training framework [12]. However, we aim at using a given φ together with its training framework as a black box, so performing SSL only at the data level.…”

Section: Related Workmentioning

confidence: 99%

Co-training for Deep Object Detection: Comparing Single-modal and Multi-modal Approaches

Gómez,

Villalonga,

López

2021

Preprint

View full text Add to dashboard Cite

Top-performing computer vision models are powered by convolutional neural networks (CNNs). Training an accurate CNN highly depends on both the raw sensor data and their associated ground truth (GT). Collecting such GT is usually done through human labeling, which is time-consuming and does not scale as we wish. This data labeling bottleneck may be intensified due to domain shifts among image sensors, which could force per-sensor data labeling. In this paper, we focus on the use of co-training, a semi-supervised learning (SSL) method, for obtaining self-labeled object bounding boxes (BBs), i.e., the GT to train deep object detectors. In particular, we assess the goodness of multi-modal co-training by relying on two different views of an image, namely, appearance (RGB) and estimated depth (D). Moreover, we compare appearance-based single-modal co-training with multi-modal. Our results suggest that in a standard SSL setting (no domain shift, a few human-labeled data) and under virtual-to-real domain shift (many virtual-world labeled data, no human-labeled data) multi-modal co-training outperforms single-modal. In the latter case, by performing GAN-based domain translation both co-training modalities are on pair; at least, when using an off-the-shelf depth estimation model not specifically trained on the translated images.

show abstract

Generating Accurate Pseudo-Labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations

Cited by 24 publications

References 12 publications

Weakly Supervised Pseudo-Label Assisted Learning for Als Point Cloud Semantic Segmentation

Weakly Supervised Pseudo-Label Assisted Learning for Als Point Cloud Semantic Segmentation

Co-Training for Deep Object Detection: Comparing Single-Modal and Multi-Modal Approaches

Co-training for Deep Object Detection: Comparing Single-modal and Multi-modal Approaches

Contact Info

Product

Resources

About