Nonlinear neuro-optimal tracking control via stable iterative Q-learning algorithm

Wei, Qinglai; Song, Ruizhuo; Sun, Qiuye

doi:10.1016/j.neucom.2015.05.075

Cited by 30 publications

(2 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Q-learning, proposed by Watkins [44,45], is a representative data-based adaptive dynamic programming algorithm. In the QL algorithm, the Q function depends on both system state and control, and updates policy through continuous observation of rewards of all state-action pairs [37]. The value of an action at any state can be defined using a Q-value, which is the sum of the immediate reward after executing action "a" at state "s" and the discounted reward from subsequent actions according to the best strategy.…”

Section: Q-learning Algorithmmentioning

confidence: 99%

“…Among the ML, the Q-learning (QL) is one of the reinforcement learning (RL) methods and a provably convergent direct optimal adaptive control algorithm [35]. Since offering the significant advantages of learning mechanisms that can ensure the inherent adaptability for a dynamic environment, QL can be used to find an optimal action-selection policy based on the historical and/or present state and action control [35][36][37], even for the completely uncertain or unknown dynamics [38]. Figuratively speaking, as a real human environment, the QL algorithm does not necessarily rely on a single agent to search the complete state-action space to obtain the optimal policy, but exchanges information, learning from the others [39].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Intelligent Control/Operational Strategies in WWTPs through an Integrated Q-Learning Algorithm with ASM2d-Guided Reward

Pang

Yang

et al. 2019

Water

View full text Add to dashboard Cite

The operation of a wastewater treatment plant (WWTP) is a typical complex control problem, with nonlinear dynamics and coupling effects among the variables, which renders the implementation of real-time optimal control an enormous challenge. In this study, a Q-learning algorithm with activated sludge model No. 2d-guided (ASM2d-guided) reward setting (an integrated ASM2d-QL algorithm) is proposed, and the widely applied anaerobic-anoxic-oxic (AAO) system is chosen as the research paradigm. The integrated ASM2d-QL algorithms equipped with a self-learning mechanism are derived for optimizing the control strategies (hydraulic retention time (HRT) and internal recycling ratio (IRR)) of the AAO system. To optimize the control strategies of the AAO system under varying influent loads, Q matrixes were built for both HRTs and IRR optimization through the pair of <max reward-action> based on the integrated ASM2d-QL algorithm. 8 days of actual influent qualities of a certain municipal AAO wastewater treatment plant in June were arbitrarily chosen as the influent concentrations for model verification. Good agreement between the values of the model simulations and experimental results indicated that this proposed integrated ASM2d-QL algorithm performed properly and successfully realized intelligent modeling and stable optimal control strategies under fluctuating influent loads during wastewater treatment.

show abstract

Section: Q-learning Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Intelligent Control/Operational Strategies in WWTPs through an Integrated Q-Learning Algorithm with ASM2d-Guided Reward

Pang

Yang

et al. 2019

Water

View full text Add to dashboard Cite

show abstract

Adaptive sliding mode control of robotic manipulator based on reinforcement learning

Ren,

Chen,

Miao

et al. 2024

Asian Journal of Control

View full text Add to dashboard Cite

Robotic manipulators usually exhibit time‐varying, nonlinear, and coupled dynamics due to the parameter perturbations, disturbances, and other uncertainties. Traditional control algorithms typically do not possess parameters' self‐adaptive learning ability, limiting the tracking performance of the robot. In order to address these issues, an adaptive sliding mode control method based on reinforcement learning (ASMRL) is proposed in this paper, where a proportional–integral sliding mode (PISM) controller is used to address the nonlinear problem in the system, and deep deterministic policy gradient (DDPG) agent is adopted to conduct the parameters' learning of the PISM controller using its adaptive characteristics and the autonomous learning ability. The simulation results illustrate that the proposed method can effectively achieve better tracking performance compared with two other control methods, demonstrating the effectiveness of the proposed approach.

show abstract