Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processes

Dornheim, Johannes; Link, Norbert

doi:10.1109/isetc.2018.8583854

Cited by 7 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to the motivating examples discussed above, recent years have seen multiobjective learning and planning methods applied across a wide range of problem domains including: distributed computing [27,124], drug and molecule design [62,214], cybersecurity [162], simulation [132], job shop scheduling [98], cognitive radio networks [100,129], satellite communications [45,63], recommender systems [78], power systems [34,35,97,193], building management [213], traffic management [70], manufacturing [36,54,80], bidding and pricing [76,207], education [151], and robotics [64]. The scope and variety of these applications supports our assertion that many important problems involve multiple objectives, and are best addressed using explicitly multi-objective methods.…”

Section: Other Topicsmentioning

confidence: 99%

A practical guide to multi-objective reinforcement learning and planning

Hayes

Rădulescu

Bargiacchi

et al. 2022

Auton Agent Multi-Agent Syst

151

View full text Add to dashboard Cite

Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.

show abstract

Section: Other Topicsmentioning

confidence: 99%

A practical guide to multi-objective reinforcement learning and planning

Hayes

Rădulescu

Bargiacchi

et al. 2022

Auton Agent Multi-Agent Syst

151

View full text Add to dashboard Cite

show abstract

“…The presented approach includes deep neural networks, which extract features and a trained reinforcement leaning algorithm that uses these features as an input to control the process in real-time. In the realm of metal forming Dornheim et al (Dornheim & Link, 2018;Dornheim et al, 2019) coupled a deep reinforcement learning algorithm with an FE simulation model to enable multi-objective optimization of a deep drawing process. The presented approach controls the blank holder force over the process in such a way that the product was manufactured as material-efficiently as possible and with as little residual stresses as possible while ensuring the desired product geometry.…”

Section: Machine Learning and Its Applications In Metal Formingmentioning

confidence: 99%

Coupling of an analytical rolling model and reinforcement learning to design pass schedules: towards properties controlled hot rolling

2023

View full text Add to dashboard Cite

Rolling is a well-established forming process employed in many industrial sectors. Although highly optimized, process disruptions can still lead to undesired final mechanical properties. This paper demonstrates advances in pass schedule design based on reinforcement learning and analytical rolling models to guarantee sound product quality. Integrating an established physical strengthening model into an analytical rolling model allows tracking the microstructure evolution throughout the process, and furthermore the prediction of the yield strength and ultimate tensile strength of the rolled sheet. The trained reinforcement learning algorithm Deep Deterministic Policy Gradient (DDPG) automatically proposes pass schedules by drawing upon established scheduling rules combined with novel rule sets to maximize the final mechanical properties. The designed pass schedule is trialed using a laboratory rolling mill while the predicted properties are confirmed using micrographs and materials testing. Due to its fast calculation time, prospectively this technique can be extended to also account for significant process disruptions such as longer inter-pass times by adapting the pass schedule online to still reach the desired mechanical properties and avoid scrapping of the material.

show abstract

“…We state that the Thresholded Lexicographic Ordering action selection from Section II-C is equivalent to the policy 6 πTLO…”

Section: Appendix Equivalence Of Tlo Formulationsmentioning

confidence: 99%

“…This includes the dynamic preferences scenario [5], where the preferences are non-stationary during application. This scenario is given for example in domains in which the optimization criteria depend on dynamic prices of open markets [4], or on changing requirements for process results [6].…”

Section: Introductionmentioning

confidence: 99%

gTLO: A Generalized and Non-linear Multi-Objective Deep Reinforcement Learning Approach

Dornheim¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

In real-world decision optimization, often multiple competing objectives must be taken into account. Following classical reinforcement learning, these objectives have to be combined into a single reward function. In contrast, multiobjective reinforcement learning (MORL) methods learn from vectors of per-objective rewards instead. In the case of multipolicy MORL, sets of decision policies for various preferences regarding the conflicting objectives are optimized. This is especially important when target preferences are not known during training or when preferences change dynamically during application. While it is, in general, straightforward to extend a single-objective reinforcement learning method for MORL based on linear scalarization, solutions that are reachable by these methods are limited to convex regions of the Pareto front. Nonlinear MORL methods like Thresholded Lexicographic Ordering (TLO) are designed to overcome this limitation. Generalized MORL methods utilize function approximation to generalize across objective preferences and thereby implicitly learn multiple policies in a data-efficient manner, even for complex decision problems with high-dimensional or continuous state spaces. In this work, we propose generalized Thresholded Lexicographic Ordering (gTLO), a novel method that aims to combine non-linear MORL with the advantages of generalized MORL. We introduce a deep reinforcement learning realization of the algorithm and present promising results on a standard benchmark for nonlinear MORL and a real-world application from the domain of manufacturing process control.

show abstract

Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processes

Cited by 7 publications

References 11 publications

A practical guide to multi-objective reinforcement learning and planning

A practical guide to multi-objective reinforcement learning and planning

Coupling of an analytical rolling model and reinforcement learning to design pass schedules: towards properties controlled hot rolling

gTLO: A Generalized and Non-linear Multi-Objective Deep Reinforcement Learning Approach

Contact Info

Product

Resources

About