Asymmetric Adaptive Fusion in a Two-Stream Network for RGB-D Human Detection

Zhang, Wenli; Guo, Xiang; Wang, Jiaqi; Wang, Ning; Chen, Kaizhen

doi:10.3390/s21030916

Cited by 5 publications

(4 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In response to the second category of questions, Zhang et al [25,26] used a visualization tool to transform the output of each network layer into a heat map image. The feature extraction capability of the network layer was then judged by visual observation of the extracted contours of the foreground targets in the output heat map of each network layer.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Pruning Method for Deep Convolutional Network Based on Heat Map Generation Metrics

Zhang

Wang

Chen

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

With the development of deep learning, researchers design deep network structures in order to extract rich high-level semantic information. Nowadays, most popular algorithms are designed based on the complexity of visible image features. However, compared with visible image features, infrared image features are more homogeneous, and the application of deep networks is prone to extracting redundant features. Therefore, it is important to prune the network layers where redundant features are extracted. Therefore, this paper proposes a pruning method for deep convolutional network based on heat map generation metrics. The ‘network layer performance evaluation metrics’ are obtained from the number of pixel activations in the heat map. The network layer with the lowest ‘network layer performance evaluation metrics’ is pruned. To address the problem that the simultaneous deletion of multiple structures may result in incorrect pruning, the Alternating training and self-pruning strategy is proposed. Using a cyclic process of pruning each model once and retraining the pruned model to reduce the incorrect pruning of network layers. The experimental results show that proposed method in this paper improved the performance of CSPDarknet, Darknet and Resnet.

show abstract

Section: Related Workmentioning

confidence: 99%

“…[23,24] used a scaling factor of the Batch Normalization function to calculate the contribution of the network structure to the final results. The second type of pruning method focuses on the subjective assessment of good or bad network structures through visual images [25][26][27]. Pruning the network structures that contribute less.…”

Section: Introductionmentioning

confidence: 99%

A Pruning Method for Deep Convolutional Network Based on Heat Map Generation Metrics

Zhang

Wang

Chen

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several methods based on two-modal fusion are used [46][47][48] to demonstrate the advantages of crowd counting in terms of day and night illumination, occlusion, and scale transformation by obtaining fused features. Twostream models [49,50] are proposed to fuse hierarchical cross-modal features to achieve fully representative shared features. In addition, there are methods [51] that explore the use of shared branches to map shared information into a common feature space.…”

Section: Related Workmentioning

confidence: 99%

Research on 24-Hour Dense Crowd Counting and Object Detection System Based on Multimodal Image Optimization Feature Fusion

Ren

2022

Scientific Programming

View full text Add to dashboard Cite

Motivation. In the environment of day and night video surveillance, in order to improve the accuracy of machine vision dense crowd counting and target detection, this paper designs a day and night dual-purpose crowd counting and crowd detection network based on multimode image fusion. Methods. Two sub-models, RGBD-Net and RGBT-Net, are designed in this paper. The depth image features and thermal imaging features are effectively fused with the features of visible light images, so that the model has stronger anti-interference characteristics and robustness to the light noise interference caused by the sudden fall of light at night. The above models use density map regression-guided detection method to complete population counting and detection. Results. The model completed daytime training and testing on MICC dataset. Through verification, the average absolute error of the model was 1.025, the mean square error was 1.521, and the recall rate of target detection was 97.11%. Night vision training and testing were completed on the RGBT-CC dataset. After verification, the average absolute error of the network was 18.16, the mean square error was 32.14, and the recall rate of target detection was 97.65%. By verifying the effectiveness of the multimode medium-term fusion network, it is found to exceed the current most advanced bimodal fusion method. Conclusion. The experimental results show that the proposed multimodal fusion network can solve the counting and detection problem in the video surveillance environment during day and night. The ablation experiment further proves the effectiveness of the parameters of the two models.

show abstract

“…For RGB-D based human detection the multi-glimpse LSTM in [5] and an asymmetric adaptive fusion two-stream network (AAFTS-net, [6]) were proposed. For real-time people detection in top-view depth images from video surveillance systems, WatchNet and its extension WatchNet++ were presented in [7] and [8].…”

Section: Introductionmentioning

confidence: 99%

NAPC: A Neural Algorithm for Automated Passenger Counting in Public Transport on a Privacy-Friendly Dataset

Seidel

Jahn

Seo

et al. 2022

IEEE Open J. Intell. Transp. Syst.

View full text Add to dashboard Cite

Real-time load information in public transport is of high importance for both passengers and service providers. Neural algorithms have shown a high performance on various object counting tasks and play a continually growing methodological role in developing automated passenger counting systems. However, the publication of public-space video footage is often contradicted by legal and ethical considerations to protect the passengers' privacy. This work proposes an end-to-end Long Short-Term Memory network with a problem-adapted cost function that learned to count boarding and alighting passengers on a publicly available, comprehensive dataset of approx. 13,000 manually annotated low-resolution 3D LiDAR video recordings (depth information only) from the doorways of a regional train. These depth recordings do not allow the identification of single individuals. For each door opening phase, the trained models predict the correct passenger count (ranging from 0 to 67) in approx. 96% of boarding and alighting, respectively. Repeated training with different training and validation sets confirms the independence of this result from a specific test set.

show abstract

Asymmetric Adaptive Fusion in a Two-Stream Network for RGB-D Human Detection

Cited by 5 publications

References 38 publications

A Pruning Method for Deep Convolutional Network Based on Heat Map Generation Metrics

A Pruning Method for Deep Convolutional Network Based on Heat Map Generation Metrics

Research on 24-Hour Dense Crowd Counting and Object Detection System Based on Multimodal Image Optimization Feature Fusion

NAPC: A Neural Algorithm for Automated Passenger Counting in Public Transport on a Privacy-Friendly Dataset

Contact Info

Product

Resources

About