Railway Track Inspection Using Deep Learning Based on Audio to Spectrogram Conversion: An on-the-Fly Approach

Abstract

The periodic inspection of railroad tracks is very important to find structural and geometrical problems that lead to railway accidents. Currently, in Pakistan, rail tracks are inspected by an acoustic-based manual system that requires a railway engineer as a domain expert to differentiate between different rail tracks’ faults, which is cumbersome, laborious, and error-prone. This study proposes the use of traditional acoustic-based systems with deep learning models to increase performance and reduce train accidents. Two convolutional neural networks (CNN) models, convolutional 1D and convolutional 2D, and one recurrent neural network (RNN) model, a long short-term memory (LSTM) model, are used in this regard. Initially, three types of faults are considered, including superelevation, wheel burnt, and normal tracks. Contrary to traditional acoustic-based systems where the spectrogram dataset is generated before the model training, the proposed approach uses on-the-fly feature extraction by generating spectrograms as a deep learning model’s layer. Different lengths of audio samples are used to analyze their performance with each model. Each audio sample of 17 s is split into 3 variations of 1.7, 3.4, and 8.5 s, and all 3 deep learning models are trained and tested against each split time. Various combinations of audio data augmentation are analyzed extensively to investigate models’ performance. The results suggest that the LSTM with 8.5 split time gives the best results with the accuracy of 99.7%, the precision of 99.5%, recall of 99.5%, and F1 score of 99.5%.

IoT Based Railway Track Faults Detection and Localization Using Acoustic Analysis

“…Both algorithms have 97% accuracy in detecting the aforementioned faults. [50] developed a railway track inspection system that combines standard acoustic methods with deep learning models to improve performance. The system employs two CNN models, convolutional 1D and convolutional 2D, as well as one recurrent neural network (RNN) model, a long short-term memory (LSTM).…”

“…Additionally, the maintenance of such systems requires the employment of specialized personnel. [48,49] used audio recordings of railway line faults, while [50] used the spectrogram of audio data obtained by [49]. Only three faults were detected in [48], whereas two faults were detected in [48,49].…”

“…Only three faults were detected in [48], whereas two faults were detected in [48,49]. [50,51] used AE to detect cracks in railway tracks. Although [50,51] achieved good accuracy, but sensor to record AE has to be place on railway track and due to the complicated geometry of the rail section as well as the high cost of sensors to record AE, it is impossible to identify cracks without the proper sensor placement on railway line.…”

“…A comparison was performed with acoustic based systems. [48][49][50] presented the acoustic based railway fault detection systems. Although the system presented in this manuscript is also used acoustic data collected at main railway line.…”

Aerial Footage Analysis Using Computer Vision for Efficient Detection of Points of Interest Near Railway Tracks

“…Computer vision and deep learning methods, along with traditional ultrasonic and acceleration detection methods, are proposed to evaluate the damage to the rail surface that can drastically improve the efficiency of the detection system while reducing inspection costs. A similar approach is explored by M. Hashmi et al in [18] where they combined traditional acoustic-based systems with deep learning models to improve performance and prevent railway accidents. In this regard, two CNN models, convolutional 1D and convolutional 2D, and one recurrent neural network (RNN) model are utilized.…”