Focal learning on stranger for imbalanced image segmentation

Abstract

It is an open issue to train effective deep network models on class imbalance datasets. In the widely used cost‐sensitive imbalanced learning methods, the costs are based on the losses or class probabilities of samples. In this paper, it is discovered that these traditional cost‐sensitive methods discard the clustering feature, and introduce the errors of annotations into costs, leading to sub‐optimal models. It is further investigated that the feature magnitude of sample, which is computed before probability and loss, not only is independent of the annotation, but also represents the familiarity degree of model with the sample. These characteristics of feature magnitude are used to guide the training and inference of model. First, the concept of stranger is proposed, which is the sample with small feature magnitude value, and the idea of focal learning on strangers (FLS) is proposed. By adding the idea of FLS into two existing cost‐sensitive methods, two novel losses are put forward: instance‐level focal stranger loss (IFSL) and class‐level focal stranger loss (CFSL). The losses can improve the aggregation features of samples within class, and reduce the negative influences of annotation errors on imbalanced learning. Second, considering the large difference of feature magnitude means between minority class and majority class in case of extreme class‐imbalance dataset, a bias determination (BD) strategy is put forward to improve classification performance during inference. The methods are applied to the tasks of image semantic segmentation and salient‐instance segmentation. The experimental results on four public semantic segmentation datasets demonstrate that IFSL can reduce the over‐fitting of model, improve the classification accuracy of rare samples, and alleviate the reliance of performance on the annotation quality. The experimental results on two public salient‐instance segmentation datasets show that CFSL makes the model have better scoring ability for salient object. Besides, the BD strategy can reduce the wrong classification caused by bias model. Therefore, the proposed methods can significantly advance image segmentation.

Cluster2Former: Semisupervised Clustering Transformers for Video Instance Segmentation

“…Clustering, in particular incorporating clustering features into the learning process, has been shown to improve image segmentation with class imbalanced datasets [17]. The precision of the segmentation at critical edges can be enhanced by using a supervised edge attention module [18].…”

Enhancing human parsing with region‐level learning

“…Previous models have treated these imbalanced classes equally, which has inevitably resulted in biased learning results. Zhao et al [19] proposed to balance feature magnitude to address the problem of imbalanced problem in semantic segmentation. Nevertheless, they ignored the imbalance between segmentation regions.…”

“…Zhao et al. [19] proposed to balance feature magnitude to address the problem of imbalanced problem in semantic segmentation. Nevertheless, they ignored the imbalance between segmentation regions.…”