High-Resolution Crowd Density Maps Generation With Multi-Scale Fusion Conditional GAN

Abstract

The major challenges for density maps estimation and accurate counting stem from the largescale variations, serious occlusions, and perspective distortions. Existing methods generally suffer from the blurred density maps, which are caused by average convolution kernel, and the ineffective estimation across different crowd scenes. In this paper, we propose a multi-scale fusion conditional generative adversarial network (MFC-GAN) that can generate high-resolution and high-quality density maps. The fusion module of MFC-GAN is embedded in a multi-scale generator and discriminator architecture with a novel adversarial loss, which is designed to guide high-resolution density maps generation. In order to address the problem of scale variation, we further propose a bidirectional fusion module. It combines deep global semantic features and shallow local information by leveraging feature maps presented in different layers of the generator. Furthermore, in order to increase the effectiveness of the multi-scale fusion, we design a cross-attention fusion module, which weights the multi-scale fused feature and learns context-aware feature maps for generating high quality density maps. The experiments on four challenging datasets show the effectiveness, feasibility and robustness of the proposed MFC-GAN.

Advanced Pedestrian State Sensing Method for Automated Patrol Vehicle Based on Multi-Sensor Fusion

“…With the advancement in sensor techniques, the current mainstream crowd density estimation is achieved through neural networks [14][15][16][17][18]. Huang et al [19] proposed a multi-scale fusion conditional generative adversarial network based on a bidirectional fusion module. The network could solve the problem of scale variation effectively.…”

Attribute-Aware Generative Design With Generative Adversarial Networks

“…Since its introduction in 2014, GAN [17] continues to attract growing interests in the deep learning community and has been applied to various domains such as computer vision [28]- [33], natural language processing [34], [35], time series synthesis [36], [37], and semantic segmentation [38], [39]. Specifically, GAN has shown significant recent success in the field of computer vision on a variety of tasks such as image generation [28], [29], image to image translation [30], [31], and image super-resolution [32], [33]. The standard GAN structure comprises two neural networks: a generator G and a discriminator D which are iteratively trained by competing against each other in a minimax game, where the generator attempts to produce realistic samples while the discriminator attempts to distinguish the fake samples from the real ones.…”