Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Abstract

The crux of semi-supervised semantic segmentation is to assign adequate pseudo-labels to the pixels of unlabeled images. A common practice is to select the highly confident predictions as the pseudo ground-truth, but it leads to a problem that most pixels may be left unused due to their unreliability. We argue that every pixel matters to the model training, even its prediction is ambiguous. Intuitively, an unreliable prediction may get confused among the top classes (i.e., those with the highest probabilities), however, it should be confident about the pixel not belonging to the remaining classes. Hence, such a pixel can be convincingly treated as a negative sample to those most unlikely categories. Based on this insight, we develop an effective pipeline to make sufficient use of unlabeled data. Concretely, we separate reliable and unreliable pixels via the entropy of predictions, push each unreliable pixel to a category-wise queue that consists of negative samples, and manage to train the model with all candidate pixels. Considering the training evolution, where the prediction becomes more and more accurate, we adaptively adjust the threshold for the reliable-unreliable partition. Experimental results on various benchmarks and training settings demonstrate the superiority of our approach over the stateof-the-art alternatives. 1

PCR: Pessimistic Consistency Regularization for Semi-Supervised Segmentation

“…CPS [6] designs a mutual learning mechanism that trains two student models with pseudo labels from each other. U2PL [47] proposes to utilize unreliable pixels by negative learning in a contrastive learning manner to make full use of the unlabeled pixels. Yi et al [53] propose to use label propagation to refine the pseudo labels.…”

“…It is worth noting that our adaptive weight is different from the weights computed by top-1 confidence used to filter out low-confidence pixels [9], [21], [36]. Those weights are small for pixels with low top-1 probability, which results in those pixels not being sufficiently used in training [47]. But our weight is only small when the prediction of a pixel is confused in the top-(K+1) categories, thus our model still uses the information that its prediction should not belong to the other C-K-1 categories.…”

“…And the formulas of convex TABLE III: The mIoU on VOC2012 LowData. Results marked by † are from [6], [47], [58]. The 'cm' is cutmix.…”

“…To mitigate this problem, some existing approaches propose to filter out low-confidence predictions and train their models by only high-confidence predictions [52], [60]. Some other works utilize low-confidence predictions by negative learning [38], [47]. These methods are based on the basic fact that an uncertain prediction usually gets confused among only a few classes instead of all classes.…”

“…Hence, they use uncertain predictions as negative samples for those unlikely classes. However, filter-based approaches cannot utilize all unlabeled pixels, resulting in insufficient pixel utilization and categorically imbalanced training [47]. For negative learningbased methods, we analyze that they still implicitly learn the pseudo-label, which means the errors in their models accumulate when the pseudo label is wrong.…”