Monocular Pedestrian Orientation Recognition Based on Capsule Network for a Novel Collision Warning System

Abstract

Pedestrians are the most vulnerable road users, with around 23% of world road traffic fatalities. To prevent such traffic collisions, the Pedestrian Collision Warning System (PCWS) alerts the driver before an imminent collision. In order to protect worldwide pedestrians, the PCWS should take into account different pedestrian crossing behaviors and different road structures, especially pedestrians with risky behavior on unstructured environments. Since oblique crossing is the usual crossing way of pedestrians with risky behaviors, recognizing the pedestrian walking direction is one of the key factors to consider. However, the existent PCWS focuses more on detecting pedestrians rather than recognizing their walking direction. This paper highlights the safety of different kinds of pedestrians by presenting a novel approach used to estimate pedestrian's orientation from a single-frame. Next, the estimated orientation is integrated into the proposed PCWS system. Our method involves Capsule Network based technique to train on the pedestrian images. For this purpose, a new pedestrian orientation dataset taken from real city-scenes named SafeRoad was created, using a single camera mounted on a moving vehicle. TUD Multiview Pedestrian and Daimler datasets are thereafter used as a benchmark to evaluate the proposed approach. Experimental results show that Capsule Network exceeds significantly the accuracy performance on the three datasets compared to Convolutional Neural Networks algorithms.

GRAN: graph recurrent attention network for pedestrian orientation classification

“…On the Cityscapes data set, we will compare with the current state-of-the-art methods, including the convolutional random projection network (CRPnet), convolutional random projection forest (CRPforest) [7] and the Capsule Network (CapsNet) proposed by Dafrallah et al [49]. The confusion matrices for these algorithms are shown in Fig.…”

“…Our body parts graph is fully covered, which facilitates the distinction between left and right, so our method has higher TPR on left and right than other algorithms. Furthermore, CapsNet in [49] cannot learn by backpropagation when computing coupling coefficients, so the model generalizes poorly when the number of images is small. Therefore, the acc is not high during verification, only 83%, which is lower than our method.…”

Design, Implementation and Experimental Investigation of a Pedestrian Street Crossing Assistance System Based on Visible Light Communications

“…The probability of a pedestrian crossing or not is generally based on the analyses of the pedestrian’s posture and on the analysis of the posture changes in multiple frame images. The performances of such systems are currently being improved with the help of neural networks [ 66 ] or adversarial feature learning [ 67 ], which are orientated to establish the current pedestrian orientation [ 68 ] and the probability of future action [ 69 ]. Such systems can predict a future pedestrian’s street crossing intention with 0.6 s in advance while providing recognition accuracies higher than 93% [ 65 ].…”