A Review of Deep Reinforcement Learning Approaches for Smart Manufacturing in Industry 4.0 and 5.0 Framework

Abstract

In this review, the industry’s current issues regarding intelligent manufacture are presented. This work presents the status and the potential for the I4.0 and I5.0’s revolutionary technologies. AI and, in particular, the DRL algorithms, which are a perfect response to the unpredictability and volatility of modern demand, are studied in detail. Through the introduction of RL concepts and the development of those with ANNs towards DRL, the potential and variety of these kinds of algorithms are highlighted. Moreover, because these algorithms are data based, their modification to meet the requirements of industry operations is also included. In addition, this review covers the inclusion of new concepts, such as digital twins, in response to an absent environment model and how it can improve the performance and application of DRL algorithms even more. This work highlights that DRL applicability is demonstrated across all manufacturing industry operations, outperforming conventional methodologies and, most notably, enhancing the manufacturing process’s resilience and adaptability. It is stated that there is still considerable work to be carried out in both academia and industry to fully leverage the promise of these disruptive tools, begin their deployment in industry, and take a step closer to the I5.0 industrial revolution.

A review of reinforcement learning based approaches for industrial demand response

“…Despite these applications, challenges such as addressing physical constraints, and the difficulties of centralizing information for single-agent RL in more complex problems were highlighted for further study. The authors in [23] review deep DRL approaches for smart manufacturing in industry 4.0 and 5.0 frameworks. The review emphasizes DRL's applicability in key manufacturing activities such as path planning, process control, scheduling, maintenance, and energy management.…”

Data-Driven Process Monitoring and Fault Diagnosis: A Comprehensive Survey

“…Supervised and unsupervised learning algorithms are widely used in process monitoring applications, while semi-supervised learning is gaining traction [230,[299][300][301]. Within the context of PSE, reinforcement learning is predominantly used in automatic process control applications [302][303][304][305], although a few applications in process monitoring have been proposed [306][307][308].…”

Reinforcement Learning with Task Decomposition and Task-Specific Reward System for Automation of High-Level Tasks

“…Moreover, since reinforcement learning is achieved through interaction with the environment, it has the advantage of not requiring gain tuning or mathematical modeling. Based on these characteristics of reinforcement learning, del Real Torres et al [20] suggest that there is potential and promise for the application of reinforcement learning algorithms in automation fields, particularly in smart factories and robotics. Despite these advantages, reinforcement learning still faces challenges in terms of adapting to complex environments, ensuring stability, and maintaining predictability, especially in complex tasks such as block stacking and pick and place during robot manipulator tasks [21].…”