Spatial Uncertainty-Aware Semi-Supervised Crowd Counting

Abstract

Semi-supervised approaches for crowd counting attract attention, as the fully supervised paradigm is expensive and laborious due to its request for a large number of images of dense crowd scenarios and their annotations. This paper proposes a spatial uncertainty-aware semi-supervised approach via regularized surrogate task (binary segmentation) for crowd counting problems. Different from existing semisupervised learning-based crowd counting methods, to exploit the unlabeled data, our proposed spatial uncertaintyaware teacher-student framework focuses on high confident regions' information while addressing the noisy supervision from the unlabeled data in an end-to-end manner. Specifically, we estimate the spatial uncertainty maps from the teacher model's surrogate task to guide the feature learning of the main task (density regression) and the surrogate task of the student model at the same time. Besides, we introduce a simple yet effective differential transformation layer to enforce the inherent spatial consistency regularization between the main task and the surrogate task in the student model, which helps the surrogate task to yield more reliable predictions and generates high-quality uncertainty maps. Thus, our model can also address the task-level perturbation problems that occur spatial inconsistency between the primary and surrogate tasks in the student model. Experimental results on four challenging crowd counting datasets demonstrate that our method achieves superior performance to the state-of-the-art semi-supervised methods. Code is available at : https://github.com/ smallmax00/SUA_crowd_counting

Reinforcing Local Feature Representation for Weakly-Supervised Dense Crowd Counting

“…IRAST [17] introduces surrogate tasks and develops a self-training method with much fewer density map annotations. SUA [19] proposes a spatial uncertainty-aware semi-supervised approach, where only some images in the training sets are labeled. Xu et al [33] proposes Partial Annotation Learning only using partial annotations in each image as training data.…”

“…Our method decreases the MAE metric by 9.6%, 15.8% and 25.9% respectively on the three CNN backbones, which verifies the effectiveness of our proposed method. This per- [27] 52.74 85.06 6.25 9.9 85.32 154.5 77.44 362 --IRAST(2020) [17] 86.9 148.9 14.7 22.9 135.6 233.4 ----SUA(2021) [19] 68.5 121. 9 [12,7] as the backbone.…”

“…There is a significant reduction in the annotation amount in semi-supervised methods [20,17,19,33], where only limited location annotations are used in the training process, e.g., annotations of limited images [20,17,19] or partial annotations in each image [33]. However, the location annotations are still required to supervise the network training.…”

DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification

“…However, the enormous sizes of WSIs bring along substantial burden for pixel-level annotation. This problem in turn encourages researchers to develop deep learning based models trained with limited annotations, termed as "Weakly Supervised" or "Semi-Supervised" [22,26,35,41]. A large proportion of existing weakly supervised works for WSI classification are characterized as "multiple instance learning" (MIL) [1,5,8,25].…”

Uncertainty-Aware Deep Co-training for Semi-supervised Medical Image Segmentation

“…Fortunately, semi-supervised methods leverage not only labeled data but also unlabeled data [5], freeing the researchers from labeling work. Therefore, semi-supervised learning has attracted great attention [6], [7], [8].…”