Railway infrastructure maintenance efficiency improvement using deep reinforcement learning integrated with digital twin based on track geometry and component defects

Abstract

Railway maintenance is a complex and complicated task in the railway industry due to the number of its components and relationships. Ineffective railway maintenance results in excess cost, defective railway structure and components, longer possession time, poorer safety, and lower passenger comfort. Of the three main maintenance approaches, predictive maintenance is the trendy one, and is proven that it provides the highest efficiency. However, the implementation of predictive maintenance for the railway industry cannot be done without an efficient tool. Normally, railway maintenance is corrective when some things fail or preventive when maintenance is routine. A novel approach using an integration between deep reinforcement learning and digital twin is proposed in this study to improve the efficiency of railway maintenance which other techniques such as supervised and unsupervised learning cannot provide. In the study, Advantage Actor Critic (A2C) is used to develop a reinforcement learning model and agent to fulfill the need of the study. Real-world field data over four years and 30 km. is obtained and applied for developing the reinforcement learning model. Track geometry parameters, railway component defects, and maintenance activities are used as parameters to develop the reinforcement learning model. Rewards (or penalties) are calculated based on maintenance costs and occurring defects. The new breakthrough exhibits that using reinforcement learning integrated with digital twin can reduce maintenance activities by 21% and reduce the occurring defects by 68%. Novelties of the study are the use of A2C which is faster and provides better results than other traditional techniques such as Deep Q-learning (DQN), each track geometry parameter is considered without combining into a track quality index, filed data are used to develop the reinforcement learning model, and seven independent actions are included in the reinforcement learning model. This study is the world’s first to contribute a new guideline for applying reinforcement learning and digital twins to improve the efficiency of railway maintenance, reduce the number of defects, reduce the maintenance cost, reduce the possession time for railway maintenance, improve the overall safety of the railway operation, and improve the passenger comfort which can be seen from its results.

Embedded Highway Health Maintenance System Based on Digital Twin Superposition Model

“…The bending deformation of foundation works and cement concrete pavement is transmitted vertically upward to the concrete base, and the overall structure of the road is caused by the bending deformation [11]. The deformation of flexible cement concrete pavement is large, and the foundation bearing capacity of soft soil cement concrete pavement and high fill cement concrete pavement are the factors causing damage to the foundation structure [12]. In addition, the relatively large axle load and the factors related to the vibration of the axle load and motor vehicles, such as the normal time interval, may cause such damage to the road during operation.…”

Agent-based modelling of high-speed railway interdependent critical infrastructures facing physical and cyber threats

“…Another important aspect of high-speed rail systems is its superior environmental sustainability compared to other modes of transportation. Recent studies have looked at the ways in which high-speed rail systems can reduce carbon emissions and help mitigate the negative environmental impacts of transportation (Sresakoolchai and Kaewunruen, 2023a;Sresakoolchai and Kaewunruen, 2023b). In particular, high-speed rail systems have the potential to significantly reduce emissions associated with air travel, a major contributor to climate change (Liu et al, 2021).…”

Circular economy implementation in railway systems beyond net zero

“…Figure 1 illustrates and demonstrates circular economy strategies, tactics and tools that will enable railway systems to be beyond net zero. For instance, IoT (internet of things) and digital twins of rail infrastructure systems have been proven to drive the carbon emission reduction while improving resilience over the life cycle (Kaewunruen et al, 2023;Sresakoolchai and Kaewunruen, 2023). The key solutions to overcome the challenge and barriers are that 1) the government needs to be impartial and work collaboratively with both academia and industry to develop and implement many more innovations beyond net zero; 2) fundings should be strategically reallocated to green solutions that timely and rightfully tackle the climate and carbon issues; 3) all stakeholders need to be objective and adopt inclusive, unbiased system-thinking approach to work together beyond net zero.…”