A Rail Fastener Tightness Detection Approach Using Multi-source Visual Sensor

Abstract

At present, the method of two-dimensional image recognition is mainly used to detect the abnormal fastener in the rail-track inspection system. However, the too-tight-or-too-loose fastener condition may cause the clip of the fastener to break or loose due to the high frequency vibration shock, which is difficult to detect from the two-dimensional image. In this practical application background, 3D visual detection technology provides a feasible solution. In this paper, we propose a fundamental multi-source visual data detection method, as well as an accurate and robust fastener location and nut or bolt segmentation algorithm. By combining two-dimensional intensity information and three-dimensional depth information generated by the projection of line structural light, the locating of nut or bolt position and accurate perception of height information can be realized in the dynamic running environment of railway. The experimental results show that the static measurement accuracy in the vertical direction using the structural light vision sensor is 0.1 mm under the laboratory condition, and the dynamic measurement accuracy is 0.5 mm under the dynamic train running environment. We use dynamic template matching algorithm to locate fasteners from 2D intensity map, which achieves 99.4% accuracy, then use the watershed algorithm to segment the nut and bolt from the corresponding depth image of located fastener. Finally, the 3D shape of the nut and bolt is analyzed to determine whether the nut or bolt height meets the local statistical threshold requirements, so as to detect the hidden danger of railway transportation caused by too loose or too tight fasteners.

Robotic Railway Multi-Sensing and Profiling Unit Based on Artificial Intelligence and Data Fusion

“…Visual sensing, such as two-dimensional image recognition, is also employed in the measurement and/or detection of the abnormal fastener in the rail-track inspection system. The authors of [ 15 ] propose a multi-source visual data detection method, and an accurate and robust fastener location and nut or bolt segmentation algorithm. They write that “By combining two-dimensional intensity information and three-dimensional depth information generated by the projection of line structural light, the locating of nut or bolt position and accurate perception of height information can be realized in the dynamic running environment of railway.” In addition, in a similar work, Liu [ 16 ] describes a method for foreign objects detection, based on a deep trust network for railway environment protection and trains safety.…”

A Deep Learning Approach Towards Railway Safety Risk Assessment

“…Sydney trains conducted condition monitoring for inspections and prevention of overhead wiring teardowns using laser and computer vision technologies [163]. Similarly, deep learning has been implemented to conduct traffic signal detection [164], [165], predict train delays [166], detect rail fastener defects and ballast history [167]- [169], detect cracks in and the shape and location of bolts [170], inspect railway ties [134], predict safety risks in communication-based train control systems (CBTCs) [171] and to perform subgrade status inspections [172].…”

Railway Track Vibration Analysis and Intelligent Recognition of Fastener Defects

“…In the field of vision-based detection, a variety of automatic detection algorithms for missing fasteners have been developed in the last ten years. [5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22] A multilayer perceptron was used as a classifier to detect the damage and missing of rail fasteners. [5,6] The classification of fastener status was realized through the multiple signal classification algorithm and the LDA feature extraction algorithm.…”