Reinforcement learning control of constrained dynamic systems with uniformly ultimate boundedness stability guarantee

Han, Minghao; Tian, Yuan; Zhang, Lixian; Wang, Jun; Pan, Wei

doi:10.1016/j.automatica.2021.109689

Cited by 42 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a learning control method, reinforcement learning (RL) has been a research hotspot in recent years (Han et al, 2021; Lambert et al, 2021; Zhang et al, 2021). RL aims to train the information from the environment to serve as an evaluation signal instead of the error signal (Jagodnik et al, 2017; Perrusquía et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Simultaneous locomotion and manipulation control of quadruped robots using reinforcement learning-based adaptive fractional-order sliding-mode control

Farid

2023

Transactions of the Institute of Measurement and Control

View full text Add to dashboard Cite

This paper investigates a model-free reinforcement learning-based approach that enables the quadruped robot to manipulate objects while maintaining its balance and dynamic stability during walking. At first, the dynamics of quadruped robots in two sub-spaces, position control space and force control space, are developed. Then, a new long-term performance index is introduced, and a radial basis function neural network as a critic network is presented to estimate the unobtainable long-term performance index. Based on the exported reinforcement signal, the actor neural network is introduced to generate the feedforward compensation term to cope with the nonlinear dynamics and the system uncertainties. The robustness of the actor-critic reinforcement learning algorithm is enhanced by using a fractional-order sliding-mode controller in the closed-loop system. The online adaptive laws for both the critic and actor-network weights are obtained using the Lyapunov stability theory. As a result, the uniformly ultimately boundedness of the position and the force tracking errors are proven. Finally, numerical simulations are conducted to illustrate the feasibility and effectiveness of the proposed adaptive actor-critic learning-based control scheme.

show abstract

Section: Introductionmentioning

confidence: 99%

Simultaneous locomotion and manipulation control of quadruped robots using reinforcement learning-based adaptive fractional-order sliding-mode control

Farid

2023

Transactions of the Institute of Measurement and Control

View full text Add to dashboard Cite

show abstract

“…As the rapid development in artificial intelligence in recent years has roundly impacted the traditional control field, learning-based and data-driven approaches, especially reinforcement learning (RL) and neural networks, have become a promising research tropic. Different from traditional control strategies that need to make assumptions based on the dynamics model [ 19 , 20 ], reinforcement learning can directly learn the policy by interacting with the system. Back in 2005, Adda et al presented a reinforcement learning algorithm for learning control of stochastic micromanipulation systems [ 21 ].…”

Section: Introductionmentioning

confidence: 99%

Adaptive Sliding Mode Disturbance Observer and Deep Reinforcement Learning Based Motion Control for Micropositioners

Liang

Xiao

et al. 2022

Micromachines

View full text Add to dashboard Cite

The motion control of high-precision electromechanitcal systems, such as micropositioners, is challenging in terms of the inherent high nonlinearity, the sensitivity to external interference, and the complexity of accurate identification of the model parameters. To cope with these problems, this work investigates a disturbance observer-based deep reinforcement learning control strategy to realize high robustness and precise tracking performance. Reinforcement learning has shown great potential as optimal control scheme, however, its application in micropositioning systems is still rare. Therefore, embedded with the integral differential compensator (ID), deep deterministic policy gradient (DDPG) is utilized in this work with the ability to not only decrease the state error but also improve the transient response speed. In addition, an adaptive sliding mode disturbance observer (ASMDO) is proposed to further eliminate the collective effect caused by the lumped disturbances. The micropositioner controlled by the proposed algorithm can track the target path precisely with less than 1 μm error in simulations and actual experiments, which shows the sterling performance and the accuracy improvement of the controller.

show abstract

“…As the rapid development in artificial intelligence in recent years have roundly impacted the traditional control field, learning-based and data-driven approaches, especially reinforcement learning (RL) and neural networks, have become a promising research tropic. Different from traditional control strategies that need to make assumption on dynamics model [13] [14], reinforcement learning can directly learn the policy by interacting with the system. Back in 2005, Adda presented a reinforcement learning algorithm for learning control of stochastic micromanipulation systems [15].…”

Section: Introductionmentioning

confidence: 99%

Adaptive Sliding Mode Disturbance Observer and Deep Reinforcement Learning based Robust Motion Control for Micropositioners

Liang¹,

Xi²,

Xiao³

et al. 2022

Preprint

View full text Add to dashboard Cite

The robust control of high precision electromechanical systems, such as micropositioners, is challenging in terms of the inherent high nonlinearity, the sensitivity to external interference, and the complexity of accurate identification of the model parameters. To cope with these problems, this work investigates a disturbance observer-based deep reinforcement learning control strategy to realize high robustness and precise tracking performance. Reinforcement learning has shown great potential as optimal control scheme, however, its application in micropositioning systems is still rare. Therefore, embedded with the integral differential compensator (ID), deep deterministic policy gradient (DDPG) is utilized in this work with the ability to not only decrease the state error but also improves the transient response speed. In addition, an adaptive sliding mode disturbance observer (ASMDO) is proposed to further eliminate the collective effect caused by the lumped disturbances. The sterling performance is revealed with intensive tracking simulation experiments and demonstrates the improvement in the accuracy and response time of the controller.

show abstract

Reinforcement learning control of constrained dynamic systems with uniformly ultimate boundedness stability guarantee

Cited by 42 publications

References 19 publications

Simultaneous locomotion and manipulation control of quadruped robots using reinforcement learning-based adaptive fractional-order sliding-mode control

Simultaneous locomotion and manipulation control of quadruped robots using reinforcement learning-based adaptive fractional-order sliding-mode control

Adaptive Sliding Mode Disturbance Observer and Deep Reinforcement Learning Based Motion Control for Micropositioners

Adaptive Sliding Mode Disturbance Observer and Deep Reinforcement Learning based Robust Motion Control for Micropositioners

Contact Info

Product

Resources

About