Deep reinforcement learning in production systems: a systematic literature review

Abstract

Shortening product development cycles and fully customisable products pose major challenges for production systems. These not only have to cope with an increased product diversity but also enable high throughputs and provide a high adaptability and robustness to process variations and unforeseen incidents. To overcome these challenges, deep Reinforcement Learning (RL) has been increasingly applied for the optimisation of production systems. Unlike other machine learning methods, deep RL operates on recently collected sensor-data in direct interaction with its environment and enables real-time responses to system changes. Although deep RL is already being deployed in production systems, a systematic review of the results has not yet been established. The main contribution of this paper is to provide researchers and practitioners an overview of applications and to motivate further implementations and research of deep RL supported production systems. Findings reveal that deep RL is applied in a variety of production domains, contributing to data-driven and flexible processes. In most applications, conventional methods were outperformed and implementation efforts or dependence on human experience were reduced. Nevertheless, future research must focus more on transferring the findings to real-world systems to analyse safety aspects and demonstrate reliability under prevailing conditions.

Interpretable Input-Output Hidden Markov Model-Based Deep Reinforcement Learning for the Predictive Maintenance of Turbofan Engines

“…This further allows for a more efficient DRL training, since the training dataset is restricted to the imminent-to-failure states. Such agents can be deployed under situation-dependent adaptations as mentioned in (Panzer & Bender, 2021). Beyond the performance considerations of the model, the IOHMM component provides a level of interpretability in terms of identifying failure states (leading towards RUL estimation), root cause of failure, and health degradation stages.…”

Movement Analytics: Current Status, Application to Manufacturing, and Future Prospects from an AI Perspective

“…As for industrial applications, RL has been deployed in production systems for process optimization and reducing reliance on human experience. Panzer and Bender [188] conducted a literature review on this topic. (Deep) RL has been applied to robot motion problems, see, e.g., [147,136].…”

Review of Learning-Based Robotic Manipulation in Cluttered Environments

“…This review paper focuses on reviewing and analyzing deep RL-based robotics manipulation challenges in a cluttered environment. This is in contrast to other reviews, which discuss the current state of deep RL-based robotic manipulation in different areas, such as robotic manipulation [ 22 ], robotic grasping [ 21 ], pick-and-place operations [ 23 ], production systems [ 24 ], and bin picking approaches [ 25 ]. Object manipulation in cluttered environments continues to be a major unaddressed challenge, despite the enthusiasm of the scientific community and its practical importance.…”