Deep learning based virtual point tracking for real-time target-less dynamic displacement measurement in railway applications

Abstract

In the application of computer-vision-based displacement measurement, an optical target is usually required to prove the reference. If the optical target cannot be attached to the measuring objective, edge detection and template matching are the most common approaches in target-less photogrammetry. However, their performance significantly relies on parameter settings. This becomes problematic in dynamic scenes where complicated background texture exists and varies over time. We propose virtual point tracking for real-time target-less dynamic displacement measurement, incorporating deep learning techniques and domain knowledge to tackle this issue. Our approach consists of three steps: 1) automatic calibration for detection of region of interest; 2) virtual point detection for each video frame using deep convolutional neural network; 3) domain-knowledge based rule engine for point tracking in adjacent frames. The proposed approach can be executed on an edge computer in a real-time manner (i.e. over 30 frames per second). We demonstrate our approach for a railway application, where the lateral displacement of the wheel on the rail is measured during operation. The numerical experiments have been performed to evaluate our approach's performance and latency in a harsh railway environment with dynamic complex backgrounds. We make our code and data available at https://github. com/quickhdsdc/Point-Tracking-for-Displacement-Measurement-in-Railway-Applications.

Metro Wheel Set Size Measurement Method Based on CDD Image Recognition

“…During the detection process, it is necessary to design a special load circuit for the wheel set. When the subway passes, the circuit can transmit the load information in the vertical direction of the wheel to the core processing module of the system, calculate the impact load value of the wheel set, and compare the calculated value with the reference value in the database to judge whether the wear of the wheel set exceeds the specified threshold [8].…”

Automated machine learning recognition to diagnose flood resilience of railway switches and crossings

“…Other examples of the use of ABAs in the railway maintenance purpose are In 2021, the use of ABAs to detect and classify the severity of wheelflats 23 , to detect and classify the railway combined defects 24 , different railway defect detection [25][26][27][28] , defect severity classification [29][30][31][32][33] or railway operation 34,35 . Moreover, machine learning also has the potential in different areas such as geoengineering and geoscience [36][37][38][39] .…”

Bridge Deformation Measurement Using Unmanned Aerial Dual Camera and Learning-Based Tracking Method

“…One-stage networks and two-stage networks are two types of anchor-based networks. Te former regresses the class probability and location coordinate values of objects directly through the backbone network, while the latter frst generates a series of sample bounding boxes by clustering algorithms or region proposal network (RPN) and then classifes the samples through a convolutional neural network [40][41][42][43][44][45]. Object detection is divided into two parts in two-stage networks: producing region suggestions from pictures and generating fnal object frames from region proposals.…”

“…Object detection is divided into two parts in two-stage networks: producing region suggestions from pictures and generating fnal object frames from region proposals. Representative networks include faster region-convolutional neural networks (R-CNNs) [39] and feature pyramid network (FPN) [41]. Te one-stage network eliminates the requirement for an area proposal stage and generates the object's category probability and location coordinate value directly.…”