Deep reinforcement learning with shallow controllers: An experimental application to PID tuning

Lawrence, Nathan P.; Forbes, Michael G.; Loewen, Philip D.; McClement, Daniel G.; Backström, Johan U.; Gopaluni, R. Bhushan

doi:10.1016/j.conengprac.2021.105046

Cited by 66 publications

(15 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section introduces reinforcement learning as a tool to control and optimise chemical processes. While PID and model predictive (MPC) controllers dominate industrial practice, reinforcement learning is an attractive alternative, 29,176 as it has the potential to outperform existing techniques in a variety of applications, such as online optimization and control of batch processes. 177 We only discuss model-free reinforcement learning here, as model-based reinforcement learning is very closely related to data-driven MPC for chemical process applications, and a full discussion on this topic is out of the scope of this section.…”

Section: Reaction Chemistry and Engineering Reviewmentioning

confidence: 99%

Industrial data science – a review of machine learning applications for chemical and process industries

et al. 2022

View full text Add to dashboard Cite

show abstract

Section: Reaction Chemistry and Engineering Reviewmentioning

confidence: 99%

Industrial data science – a review of machine learning applications for chemical and process industries

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Although RL has been demonstrated for sequential decision making in a number of case studies [21,22,23,24], its application to physical production systems has been relatively limited. For example, [25] applied deep Q networks to optimize a flexible jobshop (i.e.…”

Section: Online Production Scheduling and Reinforcement Learningmentioning

confidence: 99%

Distributional Reinforcement Learning for Scheduling of Chemical Production Processes

Mowbray¹,

Zhang²,

Chanona³

2022

Preprint

View full text Add to dashboard Cite

Reinforcement Learning (RL) has recently received significant attention from the process systems engineering and control communities. Recent works have investigated the application of RL to identify optimal scheduling decision in the presence of uncertainty. In this work, we present a RL methodology tailored to efficiently address production scheduling problems in the presence of uncertainty. We consider commonly imposed restrictions on these problems such as precedence and disjunctive constraints which are not naturally considered by RL in other contexts. Additionally, this work naturally enables the optimization of risk-sensitive formulations such as the conditional value-at-risk (CVaR), which are essential in realistic scheduling processes. The proposed strategy is investigated thoroughly in a parallel batch production environment, and benchmarked against mixed integer linear programming (MILP) strategies. We show that the policy identified by our approach is able to account for plant uncertainties in online decision-making, with expected performance comparable to existing MILP methods. Additionally, the framework gains the benefits of optimizing for risk-sensitive measures, and identifies online decisions orders of magnitude faster than the most efficient optimization approaches. This promises to mitigate practical issues and ease in handling realizations of process uncertainty in the paradigm of online production scheduling.

show abstract

“…With the development of computer technology, it was also further enhanced to fuzzy control [ 2 , 3 , 4 , 5 , 6 ]. Today, intelligent control means, in addition to fuzzy control, a much broader class of control techniques that use various artificial intelligence (AI) computing approaches like neural networks [ 7 , 8 , 9 , 10 , 11 , 12 ], Bayesian probability [ 13 , 14 ], particle swarm optimization [ 15 , 16 , 17 ], machine learning, reinforcement learning [ 18 , 19 , 20 ], evolutionary computation, or genetic algorithms [ 21 , 22 , 23 , 24 ]. Of course, this calculation is not definitive and can be expected to grow further, with the aim to propose new solutions (as in, e.g., [ 25 , 26 ]) satisfying the continuously increasing new requirements of practice.…”

Section: Introductionmentioning

confidence: 99%

Performance Portrait Method: An Intelligent PID Controller Design Based on a Database of Relevant Systems Behaviors

Huba

Vrančić

2022

Sensors

View full text Add to dashboard Cite

The article deals with a computer-supported design of optimal and robust proportional-integral-derivative controllers with two degrees of freedom (2DoF PID) for a double integrator plus dead-time (DIPDT) process model. The particular design steps are discussed in terms of intelligent use of all available information extracted from a database of control tracking and disturbance rejection step responses, assessed by means of speed and shape-related performance measures of the process input and output signals, and denoted as a performance portrait (PP). In the first step, the performance portrait method (PPM) is used as a verifier, for whether the pilot analytical design of the parallel 2DoF PID controller did not omit practically interesting settings and shows that the optimality analysis can easily be extended to the series 2DoF PID controller. This is important as an explicit observer of equivalent input disturbances based on steady-state input values of ultra-local DIPDT models, while the parallel PID controller, allowing faster transient responses, needs an additional low-pass filter when reconstructed equivalent disturbances are required. Next, the design efficiency and conciseness in analyzing the effects of different loop parameters on changing the optimal processes are illustrated by an iterative use of PPM, enabled by the visualization of the dependence between the closed-loop performance and the shapes of the control signals. The main contributions of the paper are the introduction of PPM as an intelligent method for controller tuning that mimics an expert with sufficient experience to select the most appropriate solution based on a database of known solutions. In doing so, the analysis in this paper reveals new, previously undiscovered dimensions of PID control design.

show abstract

Deep reinforcement learning with shallow controllers: An experimental application to PID tuning

Cited by 66 publications

References 31 publications

Industrial data science – a review of machine learning applications for chemical and process industries

Industrial data science – a review of machine learning applications for chemical and process industries

Distributional Reinforcement Learning for Scheduling of Chemical Production Processes

Performance Portrait Method: An Intelligent PID Controller Design Based on a Database of Relevant Systems Behaviors

Contact Info

Product

Resources

About