Towards Locally Consistent Object Counting with Constrained Multi-stage Convolutional Neural Networks

Abstract

High-density object counting in surveillance scenes is challenging mainly due to the drastic variation of object scales. The prevalence of deep learning has largely boosted the object counting accuracy on several benchmark datasets. However, does the global counts really count? Armed with this question we dive into the predicted density map whose summation over the whole regions reports the global counts for more in-depth analysis. We observe that the object density map generated by most existing methods usually lacks of local consistency, i.e., counting errors in local regions exist unexpectedly even though the global count seems to well match with the ground-truth. Towards this problem, in this paper we propose a constrained multi-stage Convolutional Neural Networks (CNNs) to jointly pursue locally consistent density map from two aspects. Different from most existing methods that mainly rely on the multi-column architectures of plain CNNs, we exploit a stacking formulation of plain CNNs. Benefited from the internal multi-stage learning process, the feature map could be repeatedly refined, allowing the density map to approach the ground-truth density distribution. For further refinement of the density map, we also propose a grid loss function. With finer local-region-based supervisions, the underlying model is constrained to generate locally consistent density values to minimize the training errors considering both the global and local counts accuracy. Experiments on two widely-tested object counting benchmarks with overall significant results compared with state-of-the-art methods demonstrate the effectiveness of our approach.

Partial Feature Aggregation Network for Real-Time Object Counting

“…The GAME (the lower the better) metric is adopted to evaluate the model performance following the experimental setting 3) Params Hydra-3s [19] 11.0 13.7 16.7 19.3 0.93M MCNN [12,20] 7.5 9.1 11.5 15.9 0.15M AMDCN [21] 9.8 13.3 15.0 15.9 0.33M C-CNN [16] 5.7 8.0 10.8 14.6 0.073M PFANet(Ours) 3.7 5.5 7.6 10.9 0.040M CMS-CNN-3 [22] 7.2 9.7 11.4 13.5 1.03M FCNN-skip [20] 4.6 8.4 11.1 16.1 2.80M CSRNet [5] 3.6 5.6 8.6 15.0 16.26M ADCrowdNet [23] 2.4 4.1 6.8 13.6 26.02M -G stands for GAME.…”

“…Four light-weight networks (Hydra-3s [19], MCNN [12,20], AMDCN [21], and C-CNN [16]) and some of previous state-of-the-art large networks (CMS-CNN-3 [22], FCNN-skip [20], CSRNet [5], and ADCrowdNet [23]) are involved in the comparison. The results are reported in the Table 2, where lowest values are in bold in two parts.…”

A deep learning counting model applied to quality control

“…The most popular applications on large number of objects are in crowd counting and agriculture uses. Zhao et al (2020) present a powerful CNN-based application for crowd counting using density maps. The architecture includes a sequence of three sets of CNN attached with FPN without connections between layers on both networks.…”