Reinforcement learning–based fault-tolerant control with application to flux cored wire system

Lin

2020

IEEE Access

Self Cite

It is noticed that offline-training and online-implementation method is dominant in the datadriven control. However, the inconsistence existing in offline data and online data may degrade the control performance. To address the aforementioned issue, an online control strategy is developed so that the control parameters can be updated online based on the real-time data measured to ensure satisfactory control performance in this study. Specifically, an online control algorithm is addressed to control the pressing-down speed of the forging machine based on the framework of the reinforcement learning that has a capability of building a complete mapping from state space to action space only according to the neighbour samples. Rather than using the way of trials and errors which is too slow to be online implementation, a taboo search is addressed to speed up the learning-working process by directly searching the control on the current states, followed by the stability conditions, derived from Lyapunov stability theory. A coarse model that is limited to get the cost information of the reinforcement learning is used to make the best of mechanism information, which prevents the occurrence of the invalid states that do not conform to system characteristics. The effectiveness of the algorithm is demonstrated by an ultra-low forging machine, which outperforms the conventional approaches such as PID and neural network control approaches.

“…Remark 2: The optimal control ( ) can be obtained only using the state information and the immediate cost because there are only ( ), ( + 1) and in formulas (22) and (23).…”

Section: A Reinforcement Learningmentioning

confidence: 99%

“…Reference [18] presented a brief survey on the advances that have occurred in the area of deep learning. From engineering application aspect, the RL/DRL showed an excellent performance after a good training in UAV [19], air-conditioning refrigeration [20], smart power control [21], fault tolerant control [22,23] and so forth [24,25].…”

Section: Introductionmentioning

confidence: 99%

An Online Control Approach for Forging Machine Using Reinforcement Learning and Taboo Search

Lin

2020

IEEE Access

Self Cite

“…c2 = c2 + ∆ c2 (21) For an action network, a1 is denoted as the connected weight between the input layer and hidden layer, a2 as the connected weigh between the hidden layer and output layer, sigmoid function as the activation function with the same form of formula (12). One will update the connected weights a1 and a2 of the action network according to the formulas (22) and 23:…”

Section: Figure 1 Schematic Block-diagram Of the Caftcmentioning

confidence: 99%

“…Recently, RL based FTC have paid attention and some results were reported such as FTC tracking control for MIMO discretetime systems [19], FTC design for a class of nonlinear MIMO discrete-time systems [20] and RL-based FTC for linear discrete-time dynamic systems [21]. It is noticed that learning of RL is a very slow process which makes RL be limited to a small FTC system [22] although RL has succeeded in game, computer vision [23] and robot [24]. Several methods accelerating on RL which aim to a huge system have proposed such as trajectory trace [18], trust region policy optimization(TRPO) [25], actor-critic with experience replay (ACER) [26], proximal policy optimization algorithm(PPO) [27], FPGA [28] and so forth.…”

Section: Introductionmentioning

confidence: 99%

Fault Tolerant Control Using Reinforcement Learning and Particle Swarm Optimization

2020

IEEE Access

Self Cite

Diversity, uncertainty and suddenness of unexpected faults bring a challenge for fault tolerant control due to the lack of valid data especially for a fault during an early stage. In this study, a reinforcement learning approach with a critic action architecture is proposed to overcome this difficulty by designing an online learning fault-tolerant controller so that the faulty system can approximate the performance index of the fault-free system. Different from the traditional Hebb enhancement rules in the reinforcement learning, the training process is speeded by introducing a supervisory learning on the basis of the training dataset which is built with the states and the virtual optimal control acquired by particle swarm optimization. The effectiveness of the algorithm is demonstrated by a test bed of a three-tank system.

“…In this way, the RL can achieve optimal action based on the current states and environment. Along with incredible success in games [8], the RL has attracted great interest in various industries, such as robot [9,10], fault detection [11], and fault-tolerant control [12,13].…”

Section: Background and Motivationmentioning

confidence: 99%

Improvement of Refrigeration Efficiency by Combining Reinforcement Learning with a Coarse Model

2019

Processes

Self Cite

It is paramount to improve operational conversion efficiency in air-conditioning refrigeration. It is noticed that control efficiency for model-based methods highly relies on the accuracy of the mechanism model, and data-driven methods would face challenges using the limited collected data to identify the information beyond. In this study, a hybrid novel approach is presented, which is to integrate a data-driven method with a coarse model. Specifically, reinforcement learning is used to exploit/explore the conversion efficiency of the refrigeration, and a coarse model is utilized to evaluate the reward, by which the requirement of the model accuracy is reduced and the model information is better used. The proposed approach is implemented based on a hierarchical control strategy which is divided into a process level and a loop level. The simulation of a test bed shows the proposed approach can achieve better conversion efficiency of refrigeration than the conventional methods.