Reinforcement learning approach for optimal control of multiple electric locomotives in a heavy-haul freight train:A Double-Switch-Q-network architecture

Tang, Huiyue; Wang, Yuan; Liu, Xiang; Feng, Xiaoyun

doi:10.1016/j.knosys.2019.105173

Cited by 33 publications

(9 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(1) Strong adaptability for different constraints and objectives: The agent of reinforcement learning performs prediction and optimization through interaction with the environment, and it learns its "knowledge" or "experience" about the environment from sampled data rather than the prior knowledge obtained from other simulation models. Therefore, this approach has strong adaptability for different constraints and objectives, such as the control of multiple electric locomotives (Tang et al, 2020), robot pathfinding (Tozer et al, 2017), and flight taxi-out time prediction (Balakrishna et al, 2010).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A deep reinforcement learning approach to mountain railway alignment optimization

Gao

Schonfeld

et al. 2021

Computer aided Civil Eng

Self Cite

View full text Add to dashboard Cite

The design and planning of railway alignments is the dominant task in railway construction. However, it is difficult to achieve self-learning and learning from human experience with manual as well as automated design methods. Also, many existing approaches require predefined numbers of horizontal points of intersection or vertical points of intersection as input. To address these issues, this study employs deep reinforcement learning (DRL) to optimize mountainous railway alignments with the goal of minimizing construction costs. First, in the DRL model, the state of the railway alignment optimization environment is determined, and the action and reward function of the optimization agent are defined along with the corresponding alignment constraints. Second, we integrate a recent DRL algorithm called the deep deterministic policy gradient with optional human experience to obtain the final optimized railway alignment, and the influence of human experience is demonstrated through a sensitivity analysis. Finally, this methodology is applied to a real-world case study in a mountainous region, and the results verify that the DRL approach used here can automatically explore and optimize the railway alignment, decreasing the construction cost by 17.65% and 7.98%, compared with the manual alignment and with the results of a method based on the distance transform, respectively, while satisfying various alignment constraints. INTRODUCTIONThe planning and design of railway alignments is not only the foundation of railway construction but also an extensive and systematic task. The direction of a railway alignment directly affects the difficulty, cost, and safety of the railway construction and operation. Therefore, the final railway alignment design should not only consider a series of natural factors such as geology and topography in the © 2021 Computer-Aided Civil and Infrastructure Engineering railway area but also satisfy other constraints, including those regarding the existing railway, historical sites, and environmental protection zones in the target area. Overall, pathfinding for a railway alignment is an optimization and decision-making problem involving many restrictive factors (Li et al., 2013). Traditional railway path planning is performed manually. Based on work experience and accumulated knowledge, designers analyze, evaluate, and compare multiple Comput Aided Civ Inf. 2022;37:73-92.wileyonlinelibrary.com/journal/mice ing requirements, thus ensuring that the horizontal alignment satisfies the constraints in the horizontal plane. In this study, we set 𝑅 𝑖 as a fixed value, which should exceed the minimum allowed value of 600 m (specified in Table 1), to fit the horizontal circular curve.

show abstract

Section: Methodsmentioning

confidence: 99%

“…The action values (also called Q-values) under each state are updated when the agent comes across the corresponding state-action pairs. The principle of Q-learning (Tang et al, 2020) is presented as follows.…”

Section: Basic Principles Of Q-learning and Deep Q-networkmentioning

confidence: 99%

A deep reinforcement learning approach to mountain railway alignment optimization

Gao

Schonfeld

et al. 2021

Computer aided Civil Eng

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, an 18-dimensional array {X 1 , X 2 , • • • X 17 , Y} consisting of feature sets and class labels is obtained. The specific processes of modeling are shown as follows [27,28]:…”

Section: Modeling Of Air Braking For Heavy-haul Trains Based On the A...mentioning

confidence: 99%

An AdaBoost-Based Intelligent Driving Algorithm for Heavy-Haul Trains

et al. 2021

View full text Add to dashboard Cite

Heavy-haul trains have the characteristics of large volume, long formation, and complex line conditions, which increase the driving difficulty of drivers and can easily cause safety problems. In order to improve the safety and efficiency of heavy-haul railways, the train control mode urgently needs to be developed towards the direction of automatic driving. In this paper, we take the Shuohuang Railway as the research background and analyze the train operation data of SS4G locomotives. We find that the proportion of operation data under different working conditions is seriously out of balance. Aiming at this unbalanced characteristic, we introduce the classification method in the field of machine learning and design an intelligent driving algorithm for heavy-haul trains. Specifically, we extract the data by random forest algorithm and compare the classification performance of C4.5 and CART algorithms. We then select the CART algorithm as the base classifier of the AdaBoost algorithm to build the model of the automatic air brake. For the purpose of heightening the precision of the model, we optimize the AdaBoost algorithm by improving the generation of training subsets and the weight of voting. The numerical results certify the effectiveness of our proposed approach.

show abstract

“…Based on integral reinforcement learning and parameter identification methods, an adaptive control scheme was proposed in [18] to achieve the tracking control of high-speed trains. In [19], a double-switch Q-network architecture was proposed for the optimal control of a heavyhaul freight train. In the aforementioned reinforcement learning based controller design, the reinforcement learning model was trained offline in a simulation environment generally.…”

Section: Introductionmentioning

confidence: 99%

Data-Driven Koopman Model Predictive Control for Optimal Operation of High-Speed Trains

et al. 2021

View full text Add to dashboard Cite

Automatic train operation systems of high-speed trains are critical to guarantee operational safety, comfort, and parking accuracy. However, implementing optimal automatic operation control is challenging due to the train's uncertain dynamics and actuator saturation. To address this issue, this paper develops a data-driven Koopman model based predictive control method for automatic train operation systems. The proposed control scheme is designed within a data-driven framework. First, using operational data of trains and the Koopman operator, an explicit linear Koopman model is built to characterize the train dynamics. Then, a model predictive controller is designed based on the Koopman model under comfort and actuator constraints. Furthermore, an online update mechanism for the Koopman model is developed to cope with the changing dynamic characteristics of trains, which reduces the accumulation errors and improves control performance. Stability analysis of the closed-loop control system is provided. Comparative simulation results validate the effectiveness of the proposed control approach.

show abstract

Reinforcement learning approach for optimal control of multiple electric locomotives in a heavy-haul freight train:A Double-Switch-Q-network architecture

Cited by 33 publications

References 46 publications

A deep reinforcement learning approach to mountain railway alignment optimization

A deep reinforcement learning approach to mountain railway alignment optimization

An AdaBoost-Based Intelligent Driving Algorithm for Heavy-Haul Trains

Data-Driven Koopman Model Predictive Control for Optimal Operation of High-Speed Trains

Contact Info

Product

Resources

About