Deep Reinforcement Learning Approach for Material Scheduling Considering High-Dimensional Environment of Hybrid Flow-Shop Problem

Abstract

Manufacturing sites encounter various scheduling problems, which must be dealt with to efficiently manufacture products and reduce costs. With the development of smart factory technology, many elements at manufacturing sites have become unmanned and more complex. Moreover, owing to the mixing of several processes in one production line, the need for efficient scheduling of materials has emerged. The aim of this study is to solve the material scheduling problem of many machines in a hybrid flow-shop environment using deep reinforcement learning. Most previous work has ignored some conditions, which were critical for solving practical problems. Such critical conditions make the scheduling more complex and difficult to solve. They expand the size of the state and large action space and make learning in an environment with many machines problematic. In this study, a reinforcement learning approach was developed considering practical factors such as the processing time and material transfer to solve realistic manufacturing scheduling problems. Additionally, a method to simplify the high-dimensional environmental space at manufacturing sites for efficient learning was established to solve the problem of learning in a high-dimensional space. Through experiments, we showed that our approach could optimally schedule material scheduling in multi-process lines, which contributes to realistic manufacturing intelligence.

A Review on Intelligent Control Theory and Applications in Process Optimization and Smart Manufacturing

“…Table 9 showcases various RL methodologies designed to fulfill the objectives of smart manufacturing. [210] centrifugal impeller geometry optimization DDPG [211] costs chemical model development Q [212] inventory cost making ordering decisions Q [213] production efficiency service job shop scheduling Q [214] hybrid flow-shop material scheduling DQN, PPO [215] product quality robot quality inspection AC [216] laser powder bed fusion surface roughness MBPO [192] 3.4. Generative Adversarial Network…”

“…Processes 2023, 11, x FOR PEER REVIEW 17 of 34 hybrid flow-shop material scheduling DQN, PPO [215] product quality robot quality inspection AC [216] laser powder bed fusion surface roughness MBPO [192] 3.4. Generative Adversarial Network…”

Neuro-Evolution of Augmenting Topologies for Dynamic Scheduling of Hybrid Flow Shop Problem

“…Han et al first proposed a reinforcement learning method for HFSP [1]. Gil and Lee studied the use of the deep reinforcement learning approach to solve the material scheduling problem of many machines in a hybrid flow shop environment [2]. Cai et al proposed a new shuffle frog-learning algorithm with Q-learning to solve a distributed assembly hybrid flow shop scheduling problem with fabrication, transportation and assembly [3].…”