Data-Driven Robust Control for Railway Driven Independently Rotating Wheelsets Using Deep Deterministic Policy Gradient

Wei, Juyao; Lu, Zixing; Yang, Zhe; He, Yang; Wang, Xiaochao

doi:10.1007/978-3-031-07305-2_10

Cited by 4 publications

(7 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each IRW observes only its local state without considering the dynamic states of the entire vehicle or other IRWs, which reduces the observation dimensionality of each agent, achieving local controllers and reducing communication costs. The observation state vector, based on the active guidance control objectives and vehicle dynamics model shown in Equations ( 1)- (7), is defined in Equation (14).…”

Section: Local and Global Observation Spacementioning

confidence: 99%

“…Recognizing the limitations of traditional control strategies, there is a growing interest in data-driven methods. Our previous research explored the application of deep reinforcement learning (DRL)-based controllers, including the deep deterministic policy gradient (DDPG) and Ape-X DDPG [13,14], leveraging deep neural networks' ability to fit nonlinear systems. Nevertheless, several limitations are encountered: (a) the existing DRL-based controllers require multiple dynamic parameters from all IRWs, and the high dimensionality of observation spaces leads to slow convergence during training; (b) current strategies mainly focus on the centralized control of the entire vehicle without achieving local control for an individual IRW, potentially affecting computational efficiency in practical applications.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multiagent Reinforcement Learning for Active Guidance Control of Railway Vehicles with Independently Rotating Wheels

Wei,

Lu,

Yin

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

This paper presents a novel data-driven multiagent reinforcement learning (MARL) controller for enhancing the running stability of independently rotating wheels (IRW) and reducing wheel–rail wear. We base our active guidance controller on the multiagent deep deterministic policy gradient (MADDPG) algorithm. In this framework, each IRW controller is treated as an independent agent, facilitating localized control of individual wheelsets and reducing the complexity of the required observations. Furthermore, we enhance the MADDPG algorithm with prioritized experience replay (PER), resulting in the PER-MADDPG algorithm, which optimizes training convergence and stability by prioritizing informative experience samples. In this paper, we compare the PER-MADDPG algorithm against existing controllers, demonstrating the superior simulation performance of the proposed algorithm, particularly in terms of self-centering capability and curve-negotiation behavior, effectively reducing the wear number. We also develop a scaled IRW vehicle for active guidance experiments. The experimental results validate the enhanced running performance of IRW vehicles using our proposed controller.

show abstract

Section: Local and Global Observation Spacementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Multiagent Reinforcement Learning for Active Guidance Control of Railway Vehicles with Independently Rotating Wheels

Wei,

Lu,

Yin

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…The main objective was to improve the curving performance of a railway vehicle with the IRWs by applying negative tread conicity. Wei et al investigated 28 the properties of a vehicle equipped with a data-driven robust controller for the active steering of its driven IRWs. Using the deep deterministic policy gradient, these researchers aimed to improve the guidance and curve-negotiation behavior of the driving system of a vehicle with IRWs.…”

Section: The State Of the Art Of Railway Vehicles Equipped With Indep...mentioning

confidence: 99%

“…The parameters used were: Kalker weighting factor 1, minimum reference velocity 0.01 m/s, friction coefficient 0.25, stick friction coefficient 0.4, stiffness for the stick 100.106 N/m, and damping for the stick 1000 Ns/m. As it is mentioned above, there are many various researchers focused on the investigation of the properties of the wheelsets with the IRWs 28,66,67 . However, only a few works are available with the research of the railway wheelsets design with the technical solution including the independent rotating flange of a railway wheel.…”

Section: Simulation Analysis Of Railway Wheel Designs On Kinematic Ru...mentioning

confidence: 99%

Kinematic running resistance of an urban rail vehicle undercarriage: a study of the impact of wheel design

Semenov

Mikhailov

Kovtanets

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Urban railway vehicles are important means of transport in towns and cities due to their high capacity, power source, and low running resistance, which make them efficient for operation. Although these properties are considered advantages, there is still room for improvement in their operational efficiency. The main objective of this article is to investigate the impact of railway wheel design on the level of kinematic running resistance, which is expressed as the amount of mechanical energy losses during the interaction of wheels with rails. This research focuses on simulation computations of two variants of wheel design schemes: the traditional design scheme (TKS) and a perspective design scheme (PKS) characterized by a rotating flange independently of the wheel tread surface. Two undercarriage multibody models have been created, one with TKS and one with PKS, and simulation computations have been performed for running speeds of 10 km/h, 20 km/h, 30 km/h, and 40 km/h on track models in curves with radii of 20, 50, 100, 150, 200, and 250 m. The evaluated indicators affecting the level of mechanical energy losses were creep forces, slip velocities, and average power. The most important findings of this study are that the PKS design scheme resulted in lower values of all assessed parameters.

show abstract

“…In existing designs of active steering controllers for vehicles, control algorithms like PID controllers [10][11][12][13], H∞ controllers [14,15], sliding mode controllers [16], and neural-Processes 2023, 11, 2677 2 of 25 network-based controllers [17,18] have all been studied. Ahn [10] adopted a centering control approach, designed a PI controller, and conducted active steering control experiments on a small-scale roller rig using DIRWs driven by surface permanent magnet synchronous motors (SPMSMs).…”

Section: Introductionmentioning

confidence: 99%

Active Steering Controller for Driven Independently Rotating Wheelset Vehicles Based on Deep Reinforcement Learning

Lu,

Wei,

Wang

2023

Processes

Self Cite

View full text Add to dashboard Cite

This paper proposes an active steering controller for Driven Independently Rotating Wheelset (DIRW) vehicles based on deep reinforcement learning (DRL). For the two-axle railway vehicles equipped with Independently Rotating Wheelsets (IRWs), each wheel connected to a wheel-side motor, the Ape-X DDPG controller, an enhanced version of the Deep Deterministic Policy Gradient (DDPG) algorithm, is adopted. Incorporating Distributed Prioritized Experience Replay (DPER), Ape-X DDPG trains neural network function approximators to obtain a data-driven DIRW active steering controller. This controller is utilized to control the input torque of each wheel, aiming to improve the steering capability of IRWs. Simulation results indicate that compared to the existing model-based H∞ control algorithm and data-driven DDPG control algorithm, the Ape-X DDPG active steering controller demonstrates better curving steering performance and centering ability in straight tracks across different running conditions and significantly reduces wheel–rail wear. To validate the proposed algorithm’s efficacy in real vehicles, a 1:5 scale model of the DIRW vehicle and its digital twin dynamic model were designed and manufactured. The proposed control algorithm was deployed on the scale vehicle and subjected to active steering control experiments on a scaled track. The experimental results reveal that under the active steering control of the Ape-X DDPG controller, the steering performance of the DIRW scale model on both straight and curved tracks is significantly enhanced.

show abstract

Data-Driven Robust Control for Railway Driven Independently Rotating Wheelsets Using Deep Deterministic Policy Gradient

Cited by 4 publications

References 2 publications

Multiagent Reinforcement Learning for Active Guidance Control of Railway Vehicles with Independently Rotating Wheels

Multiagent Reinforcement Learning for Active Guidance Control of Railway Vehicles with Independently Rotating Wheels

Kinematic running resistance of an urban rail vehicle undercarriage: a study of the impact of wheel design

Active Steering Controller for Driven Independently Rotating Wheelset Vehicles Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About