Multi-Level Bottom-Top and Top-Bottom Feature Fusion for Crowd Counting

Abstract

Crowd counting presents enormous challenges in the form of large variation in scales within images and across the dataset. These issues are further exacerbated in highly congested scenes. Approaches based on straightforward fusion of multi-scale features from a deep network seem to be obvious solutions to this problem. However, these fusion approaches do not yield significant improvements in the case of crowd counting in congested scenes. This is usually due to their limited abilities in effectively combining the multi-scale features for problems like crowd counting. To overcome this, we focus on how to efficiently leverage information present in different layers of the network. Specifically, we present a network that involves: (i) a multilevel bottom-top and top-bottom fusion (MBTTBF) method to combine information from shallower to deeper layers and vice versa at multiple levels, (ii) scale complementary feature extraction blocks (SCFB) involving cross-scale residual functions to explicitly enable flow of complementary features from adjacent conv layers along the fusion paths. Furthermore, in order to increase the effectiveness of the multi-scale fusion, we employ a principled way of generating scale-aware ground-truth density maps for training. Experiments conducted on three datasets that contain highly congested scenes (ShanghaiTech, UCF CROWD 50, and UCF-QNRF) demonstrate that the proposed method is able to outperform several recent methods in all the datasets.

CSE-HRNet: A Context and Semantic Enhanced High-Resolution Network for Semantic Segmentation of Aerial Imagery

“…On the other hand, since high-level semantic information is not adequate for recovering detailed spatial information, feature maps from shallower layers, which have smaller semantic levels, are required to encode low-level details and spatial information to refine the coarse high-level semantic features for accurate spatial localization [53]- [55].…”

Cascaded Multi-Task Learning of Head Segmentation and Density Regression for RGBD Crowd Counting

“…Sam et al [3] improved MCNN and propose a switchable module to classify the crowd density of each patch and assign it to corresponding regressor. Sindagi et al [12] proposed a top-down and bottom-up multi-level fusion mechanism to fuse features for crowd counting. CSRNet [21] stacks dilated convolutions after VGGNet [40].…”

“…Our method follows density map regression methods. Previous density regression based works [21], [41], [12], [39] usually first extract image/patch features using a backbone network(e.g. VGG16 [40]), and then perform density regression.…”

“…However, the input of our network has two sources: RGB image and depth map. The cascaded architecture and multi-task strategy also make our method different from most regression based methods [21], [41], [12], [36].…”

“…Conventional crowd counting methods [10], [11], [12], [6] usually estimate crowd counts from RGB images or videos. The RGB crowd counting methods can be categorized into detection based methods [13], [14], [15], [16] and regression based methods [17], [18], [19], [4], [3].…”