PedJointNet: Joint Head-Shoulder and Full Body Deep Network for Pedestrian Detection

Abstract

Pedestrian detection when occlusions exist represents a great challenge in real-world applications, including urban autonomous driving and surveillance systems. However, the head-shoulder feature of pedestrians, which is more stable and less likely to be occluded than other areas of the body, can be used as a complement to full body prediction to boost pedestrian detection accuracy. In this paper, we investigate the unique features of the head-shoulder and full body features belonging to pedestrians. Then, instead of using a popular general object detection framework like R-CNN series, SSD, or YOLO, we propose a novel pedestrian detection network, called PedJointNet, that simultaneously regresses two bounding boxes to localize the head-shoulder and full body regions based on a feasible object detection backbone. Moreover, unlike the traditional strategy of keeping the weights fixed for each attribute, we design an inbuilt mechanism to dynamically and adaptively adjust the relationships of the head-shoulder and full body predictions for more accurate pedestrian localization. We validate the effectiveness of the proposed method using the CUHK-SYSU, TownCentre, and CityPersons datasets. Overall, our two-pronged prediction approach achieves excellent performance in detecting both non-occluded and occluded pedestrians, especially under circumstances involving occlusion, as compared to other state-of-the-art methods.INDEX TERMS Pedestrian detection, head-shoulder detection, adaptively adjusted weights.

From Handcrafted to Deep Features for Pedestrian Detection: A Survey

“…Zhang et al [233] proposed double anchor region proposal networks (Double Anchor RPN) to respectively detect human heads and bodies and a joint Nms to combine the detection results. Lin et al [110] built two-branch networks for head-shoulder and full body part prediction and introduce a novel adaptive fusion mechanism. Lu et al [125] proposed semantic head detection in parallel with a body branch to address intra-class occlusion.…”

Pedestrian classification on transfer learning based deep convolutional neural network for partial occlusion handling

“…The proposed architecture outperformed the existing approaches in the performance accuracy with the several challenging datasets. Lin et al [25] introduced PedJointNet, a two-branch network architecture for pedestrian recognition that specifies both the full body and the head-shoulder component. Two feature pyramid modules were combined to create the branches, one for the head-shoulder parts of pedestrians and the other for the entire parts of pedestrians.…”

Passenger and pedestrian recognition based on neural networks and deep learning in stations

“…Distinguish the background of the person to be recognized from the subject target and carry out feature recognition (see Figure 3). The pedestrian head as a local feature combined with the full body detection results can effectively improve the success rate of human body recognition and judgment with occluded objects [4] [5]. Similarly, they all need a lot of head feature identification and whole-body feature, which leads to the complexity and vast data of the feature database, causing significant pressure on the computer's computing power.…”