On the Application of Time Frequency Convolutional Neural Networks to Road Anomalies’ Identification with Accelerometers and Gyroscopes

Abstract

The detection and identification of road anomalies and obstacles in the road infrastructure has been investigated by the research community using different types of sensors. This paper evaluates the detection and identification of road anomalies/obstacles using the data collected from the Inertial Measurement Unit (IMU) installed in a vehicle and in particular from the data generated by the accelerometers’ and gyroscopes’ components. Inspired by the successes of the application of deep learning to various identification problems, this paper investigates the application of Convolutional Neural Network (CNN) to this specific problem. In particular, we propose a novel approach in this context where the time-frequency representation (i.e., spectrogram) is used as an input to the CNN rather than the original time domain data. This approach is evaluated on an experimental dataset collected using 12 different vehicles driving for more than 40 km of road. The results show that the proposed approach outperforms significantly and across different sampling rates both the application of CNN to the original time domain representation and the application of shallow machine learning algorithms. The approach achieves an identification accuracy of 97.2%. The results presented in this paper are based on an extensive optimization both of the CNN algorithm and the spectrogram implementation in terms of window size, type of window, and overlapping ratio. The accurate detection of road anomalies/obstacles could be useful to road infrastructure managers to monitor the quality of the road surface and to improve the accurate positioning of autonomous vehicles because road anomalies/obstacles could be used as landmarks.

Performance Evaluation of You Only Look Once v4 in Road Anomaly Detection and Visual Simultaneous Localisation and Mapping for Autonomous Vehicles

“…In [27], the detection and identification of road anomalies and obstacles in the road infrastructure using data collected from an Inertial Measurement Unit (IMU) installed in a vehicle is presented. The authors evaluate the use of Convolutional Neural Network (CNN) for this task, as well as the use of time-frequency representation (spectrogram) as input to the CNN instead of the original time domain data.…”

Applications of the Generalized Morse Wavelets: A Review

“…Like other studies mentioned in this review, gamma was established as 3, while the time-bandwidth product was set at 60. Moreover, the classification of vibration signals for road surface assessment applied through GMWs was proposed by Baldini et al [60]. The previous study compared the spectrogram of the STFT and the scalogram of the WT generated by a Morse wavelet to classify road surface anomalies through the processing of accelerometer signals.…”

A scalable, self‐supervised calibration and confounder removal model for opportunistic monitoring of road degradation

“…On the other hand, Baldini et al. (2020) extract a spectrogram from the time window of the gyroscope and accelerometer samples, and a CNN is used to classify them. The authors validate their approach with 12 distinct vehicles (six among them are Fiat Panda cars) and 20 loops per car on a 2‐km test track.…”

“…In addition, stability events are identified: opening and closing of the doors, driving over speed bumps, answering a call on the phone, and so forth. On the other hand, Baldini et al (2020) extract a spectrogram from the time window of the gyroscope and accelerometer samples, and a CNN is used to classify them. The authors validate their approach with 12 distinct vehicles (six among them are Fiat Panda cars) and 20 loops per car on a 2-km test track.…”