Generating High-Quality Crowd Density Maps Using Contextual Pyramid CNNs

Abstract

We present a novel method called Contextual Pyramid CNN (CP-CNN) for generating high-quality crowd density and count estimation by explicitly incorporating global and local contextual information of crowd images. The proposed CP-CNN consists of four modules: Global Context Estimator (GCE), Local Context Estimator (LCE), Density Map Estimator (DME) and a Fusion-CNN (F-CNN). GCE is a VGG-16 based CNN that encodes global context and it is trained to classify input images into different density classes, whereas LCE is another CNN that encodes local context information and it is trained to perform patch-wise classification of input images into different density classes. DME is a multi-column architecture-based CNN that aims to generate high-dimensional feature maps from the input image which are fused with the contextual information estimated by GCE and LCE using F-CNN. To generate high resolution and high-quality density maps, F-CNN uses a set of convolutional and fractionally-strided convolutional layers and it is trained along with the DME in an end-to-end fashion using a combination of adversarial loss and pixellevel Euclidean loss. Extensive experiments on highly challenging datasets show that the proposed method achieves significant improvements over the state-of-the-art methods.Recent CNN-based methods using different multi-scale architectures [50,23,29] have achieved significant success in addressing some of the above issues, especially in the high-density complex crowded scenes. However, these methods tend to under-estimate or over-estimate count in the presence of high-density and low-density crowd images, respectively (as shown in Fig. 2). A potential solution is to use contextual information during the learning process. Several recent works for semantic segmentation [21], scene parsing [51] and visual saliency [52] have demonstrated that incorporating contextual information can provide significant improvements in the results. Motivated by their success, we believe that availability of global context shall aid the learning process and help us achieve better

SCAR: Spatial-/channel-wise attention regression networks for crowd counting

“…MAE MSE Idrees et al [40] 419.5 541.6 Zhang et al [37] 467.0 498.5 MCNN [17] 377.6 509.1 Switching-CNN [18] 318.1 439.2 CP-CNN [21] 295.8 320.9 DR-ResNet [38] 307.4 421.6 CSRNet [6] 266.1 397.5 ic-CNN [39] 260.9 365.5 SCAR (ours) 259.0 374.0 and cross-location evaluation) to show the capacity from different angles.…”

“…With the development of deep learning, many CNN-based counting models [17,18,19,21,26,22,16,27] obtain good performance. Zhang et al [17] propose a multi-column FCN to encode the local features with the different kernel sizes.…”

“…However, the above methods only focus on the local appearance features so that they cannot encode the contextual information. Sindagi and Patel [21] propose a Contextual Pyramid CNN (CP-CNN) to integrate the local and global contextual information. Li et al [6] embed the dilation convolution operation into the FCN to encode the contextual features.…”

ZoomCount: A Zooming Mechanism for Crowd Counting in Static Images

“…Detection-based methods perform well in low crowd images but could not be generalized well to high-density crowd images as detection fails miserably in such cases due to very few pixels per head or person. Density map estimation methods [22], [39], [40] Left and right graphs compare ten cases each, belonging to high dense and low dense extreme respectively for Density map [3], DenseNet [20] based direct regression and our method. As shown, other models either highly underestimate or overestimate, whereas the proposed method remains the closest to the ground truth (GT) bar in most cases.…”

A Real-Time Deep Network for Crowd Counting

“…With recent development of the convolutional neural network (CNN), researchers employ CNN to accurately estimate the crowd count from images or videos [1][2][3][4][5]. However, it is always challenging to deal with scale variations on static images, especially in diversified scenes such as different camera perspectives and irregular crowd clusters.…”