Suspension Regulation of Medium-Low-Speed Maglev Trains Via Deep Reinforcement Learning

Zhao, Feiran; You, Keyou; Song, Sejun; Zhang, Wenyue; Tong, Laisheng

doi:10.1109/tai.2021.3097313

Cited by 16 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using deep learning techniques, RL agents can effectively learn and represent the relationships between states, actions, and rewards, which enables them to make informed decisions and improve their performance over time. In recent years, DRL has emerged as a highly promising approach for addressing real-world problems, such as optimal control of nonlinear systems [122], pedestrian regulation [123], and traffic grid signal control [58]. Zhao et al [27] modeled the maglev system control into a continuous Markov decision process problem (MDP) and adopted the deep deterministic policy gradient (DDPG) algorithms for levitation regulation.…”

Section: Deep Learning Algorithmsmentioning

confidence: 99%

“…Neural network-based model Machine learning (ML) and deep learning (DL) have already been used for data-driven modeling in various fields, such as robotics [56] and civil structures [57]. For the maglev system, Zhao et al [58] directly trained a DRL controller using the dataset collected from the medium-low-speed maglev trains in Changsha, China. Sun et al [26] trained a deep belief network (DBN) controller to combine with an output constraint controller to handle the unmeasurable air gap velocity and disturbance.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Review of Levitation Control Methods for Low- and Medium-Speed Maglev Systems

Zhu,

Wang,

2024

Buildings

View full text Add to dashboard Cite

Maglev transportation is a highly promising form of transportation for the future, primarily due to its friction-free operation, exceptional comfort, and low risk of derailment. Unlike conventional transportation systems, maglev trains operate with no mechanical contact with the track. Maglev trains achieve levitation and guidance using electromagnetic forces controlled by a magnetic levitation control system. Therefore, the magnetic levitation control system is of utmost importance in maintaining the stable operation performance of a maglev train. However, due to the open-loop instability and strong nonlinearity of the control system, designing an active controller with self-adaptive ability poses a substantial challenge. Moreover, various uncertainties exist, including parameter variations and unknown external disturbances, under different operating conditions. Although several review papers on maglev levitation systems and control methods have been published over the last decade, there has been no comprehensive exploration of their modeling and related control technologies. Meanwhile, many review papers have become outdated and no longer reflect the current state-of-the-art research in the field. Therefore, this article aims to summarize the models and control technologies for maglev levitation systems following the preferred reporting items for systematic reviews and meta-analysis (PRISMA) criteria. The control technologies mainly include linear control methods, nonlinear control methods, and artificial intelligence methods. In addition, the article will discuss maglev control in other scenarios, such as vehicle–guideway vibration control and redundancy and fault-tolerant design. First, the widely used maglev levitation system modeling methods are reviewed, including the modeling assumptions. Second, the principle of the control methods and their control performance in maglev levitation systems are presented. Third, the maglev control methods in other scenarios are discussed. Finally, the key issues pertaining to the future direction of maglev levitation control are discussed.

show abstract

Section: Deep Learning Algorithmsmentioning

confidence: 99%

mentioning

confidence: 99%

A Review of Levitation Control Methods for Low- and Medium-Speed Maglev Systems

Zhu,

Wang,

2024

Buildings

View full text Add to dashboard Cite

show abstract

“…Additionally, damping force does not only perform as the control input of semi‐active suspension systems but also measures energy consumption (F. Zhao et al., 2021). For exmaple, the higher the damping force is, the larger current in the actuator is needed for magneto‐rheological dampers (Wu et al., 2020).…”

Section: Drl‐based Suspension Control With External Knowledgementioning

confidence: 99%

A hierarchical framework for improving ride comfort of autonomous vehicles via deep reinforcement learning with external knowledge

Chen

Zhao

et al. 2022

Computer aided Civil Eng

View full text Add to dashboard Cite

Ride comfort plays an important role in determining the public acceptance of autonomous vehicles (AVs). Many factors, such as road profile, driving speed, and suspension system, influence the ride comfort of AVs. This study proposes a hierarchical framework for improving ride comfort by integrating speed planning and suspension control in a vehicle-to-everything environment. Based on safe, comfortable, and efficient speed planning via dynamic programming, a deep reinforcement learning-based suspension control is proposed to adapt to the changing pavement conditions. Specifically, a deep deterministic policy gradient with external knowledge (EK-DDPG) algorithm is designed for the efficient selfadaptation of suspension control strategies. The external knowledge of action selection and value estimation from other AVs are combined into the loss functions of the DDPG algorithm. In numerical experiments, real-world pavements detected in 11 districts of Shanghai, China, are applied to verify the proposed method. Experimental results demonstrate that the EK-DDPG-based suspension control improves ride comfort on untrained rough pavements by 27.95% and 3.32%, compared to a model predictive control (MPC) baseline and a DDPG baseline, respectively. Meanwhile, the EK-DDPG-based suspension control improves computational efficiency by 22.97%, compared to the MPC baseline, and performs at the same level as the DDPD baseline. This study provides a generalized and computationally efficient approach for improving the ride comfort of AVs.

show abstract

“…In fact, the use of reinforcement learning methods to construct agents to learn vibration control in complex environments has achieved remarkable results [11][12][13][14][15]. This method has been applied in many fields, such as for suspension vibration control [16,17], manipulator vibration control [18][19][20], magnetorheological damper vibration control [21,22], and other vibration control applications [23][24][25], and has made remarkable progress.…”

Section: Introductionmentioning

confidence: 99%

“…In the field of suspension vibration control [16,17], Li et al [16] integrated model-free learning with model-based safety supervision. They used the underlying system dynamics and security-related constraints, used recursive feasibility techniques to construct security sets, and integrated them into the detection system constraints of reinforcement learning.…”

Section: Introductionmentioning

confidence: 99%

Vibration Control with Reinforcement Learning Based on Multi-Reward Lightweight Networks

Shu,

He,

Qiao

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

This paper proposes a reinforcement learning method using a deep residual shrinkage network based on multi-reward priority experience playback for high-frequency and high-dimensional continuous vibration control. Firstly, we keep the underlying equipment unchanged and construct a vibration system simulator using FIR filters to ensure the complete fidelity of the physical model. Then, by interacting with the simulator using our proposed algorithm, we identify the optimal control strategy, which is directly applied to real-world scenarios in the form of a neural network. A multi-reward mechanism is proposed to assist the lightweight network to find a near-optimal control strategy, and a priority experience playback mechanism is used to prioritize the data to accelerate the convergence speed of the neural network and improve the data utilization efficiency. At the same time, the deep residual shrinkage network is introduced to realize adaptive denoising and lightweightness of the neural network. The experimental results indicate that under narrowband white-noise excitation ranging from 0 to 100 Hz, the DDPG algorithm achieved a vibration reduction effect of 12.728 dB, while our algorithm achieved a vibration reduction effect of 20.240 dB. Meanwhile, the network parameters were reduced by more than 7.5 times.

show abstract

Suspension Regulation of Medium-Low-Speed Maglev Trains Via Deep Reinforcement Learning

Cited by 16 publications

References 28 publications

A Review of Levitation Control Methods for Low- and Medium-Speed Maglev Systems

A Review of Levitation Control Methods for Low- and Medium-Speed Maglev Systems

A hierarchical framework for improving ride comfort of autonomous vehicles via deep reinforcement learning with external knowledge

Vibration Control with Reinforcement Learning Based on Multi-Reward Lightweight Networks

Contact Info

Product

Resources

About