Deep Transfer Learning for Vulnerable Road Users Detection using Smartphone Sensors Data

Abstract

As the Autonomous Vehicle (AV) industry is rapidly advancing, the classification of non-motorized (vulnerable) road users (VRUs) becomes essential to ensure their safety and to smooth operation of road applications. The typical practice of non-motorized road users’ classification usually takes significant training time and ignores the temporal evolution and behavior of the signal. In this research effort, we attempt to detect VRUs with high accuracy be proposing a novel framework that includes using Deep Transfer Learning, which saves training time and cost, to classify images constructed from Recurrence Quantification Analysis (RQA) that reflect the temporal dynamics and behavior of the signal. Recurrence Plots (RPs) were constructed from low-power smartphone sensors without using GPS data. The resulted RPs were used as inputs for different pre-trained Convolutional Neural Network (CNN) classifiers including constructing 227 × 227 images to be used for AlexNet and SqueezeNet; and constructing 224 × 224 images to be used for VGG16 and VGG19. Results show that the classification accuracy of Convolutional Neural Network Transfer Learning (CNN-TL) reaches 98.70%, 98.62%, 98.71%, and 98.71% for AlexNet, SqueezeNet, VGG16, and VGG19, respectively. Moreover, we trained resnet101 and shufflenet for a very short time using one epoch of data and then used them as weak learners, which yielded 98.49% classification accuracy. The results of the proposed framework outperform other results in the literature (to the best of our knowledge) and show that using CNN-TL is promising for VRUs classification. Because of its relative straightforwardness, ability to be generalized and transferred, and potential high accuracy, we anticipate that this framework might be able to solve various problems related to signal classification.

5G-enabled V2X communications for vulnerable road users safety applications: a review

“…It can benefit from multiple data sources, as in fuzzy systems [74,114,135] where the weather conditions, the environment features, the driver and the VRU characteristics are fed into a fuzzy engine to determine the collision risk level. Moreover, learning can intelligently make decision through massive amount of communication data, or sensors [159] and environment information through data fusion [160]. Another open direction is that reinforcement learning can also enhance resource allocation by optimizing the transmission frequency of the communication messages according to the network and the sender state.…”

Gray-Level Image Transformation of Paved Road Cracks with Metaphorical and Computational Analysis

“…Besides all these advantages, this technique is still not good, because it does not significantly improve each pixel. If there is any irregularity in capturing the image, the result will not be good, and some parts will be more bright or dark with low or high contrast [34]. Irregularity in capturing the image means that we have to keep a few parameters under consideration while capturing images to be processed.…”

Driving behavior classification at signalized intersections using vehicle kinematics: Application of unsupervised machine learning

“…A driving simulator was also used to analyze driving behavior under rainy weather in [30]. In addition to driving simulators, one can collect data using smartphone sensors [31], which can be used to investigate dangerous driving recognition [32], mode recognition [33], [34], and other applications.…”