Intelligent scheduling of discrete automated production line via deep reinforcement learning

Shi, Daming; Fan, Wenhui; Xiao, Yingying; Lin, Ting-Yu; Xing, Chi

doi:10.1080/00207543.2020.1717008

Cited by 70 publications

(30 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These assume a discrete action space, which must be determined beforehand. However, for scheduling-related problems, the action space can often be discretised according to possible transition actions such as transfer or idle (as in Shi et al (2020)). It is noticeable that in comparison to process control, even fewer approaches have been adopted in a real environment.…”

Section: Production Schedulingmentioning

confidence: 99%

Deep reinforcement learning in production systems: a systematic literature review

Panzer

Bender

2021

International Journal of Production Research

128

View full text Add to dashboard Cite

Shortening product development cycles and fully customisable products pose major challenges for production systems. These not only have to cope with an increased product diversity but also enable high throughputs and provide a high adaptability and robustness to process variations and unforeseen incidents. To overcome these challenges, deep Reinforcement Learning (RL) has been increasingly applied for the optimisation of production systems. Unlike other machine learning methods, deep RL operates on recently collected sensor-data in direct interaction with its environment and enables real-time responses to system changes. Although deep RL is already being deployed in production systems, a systematic review of the results has not yet been established. The main contribution of this paper is to provide researchers and practitioners an overview of applications and to motivate further implementations and research of deep RL supported production systems. Findings reveal that deep RL is applied in a variety of production domains, contributing to data-driven and flexible processes. In most applications, conventional methods were outperformed and implementation efforts or dependence on human experience were reduced. Nevertheless, future research must focus more on transferring the findings to real-world systems to analyse safety aspects and demonstrate reliability under prevailing conditions.

show abstract

Section: Production Schedulingmentioning

confidence: 99%

Deep reinforcement learning in production systems: a systematic literature review

Panzer

Bender

2021

International Journal of Production Research

128

View full text Add to dashboard Cite

show abstract

“…Shi et al [27] applied a DQN-based scheduling algorithm to linear, parallel, and turn-back production lines in a discrete simulation environment. The algorithm showed more stable convergence and robustness than the traditional heuristic scheduling algorithm.…”

Section: Literature Reviewmentioning

confidence: 99%

Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

Zhao¹,

Wang

Tan

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Jobshop scheduling is a classic instance in the field of production scheduling. Solving and optimizing the scheduling process of the jobshop can greatly reduce the production cost of the workshop and improve the processing efficiency, thereby improving the market competitiveness of the manufacturing industry. A reasonable scheduling algorithm in jobshop can reduce manufacturing costs, shorten product delivery time, enhance customer service quality, and improve the overall performance of the manufacturing system. The jobshop scheduling problem is transformed into a reinforcement learning problem based on the Markov decision process. The performance of the adaptive scheduling algorithm in a dynamic manufacturing environment is improved based on the Deep Q Network (DQN). In the proposed scheduling algorithm, five state features of continuous value ranges are designed for input to a Deep Neural Network (DNN), as well as ten well-known heuristic dispatching rules are selected as the action set of the DQN. In the proposed scheduling algorithm, the target network and the prediction network are used to train the parameters. The DQN uses a "softmax" function as an action selection strategy during the training process, where it selects the dispatching rules with the greatest action value as the execution action. It resolves the problem whereby the sub-optimal action value is greater than the Q value of the optimal action in the early learning stage. Futhermore, the non-optimal action is selected with a greater probability in the later learning stage. Ten typical jobshop test instances called "LA" are used as a simulation object and operated in Python. The simulation results confirm that the proposed scheduling algorithm based on DQN has better performance and universality than a single dispatching rule or traditional Q learning algorithm.INDEX TERMS dynamic scheduling, deep Q network, deep reinforcement learning, dispatching rules, job shop scheduling.

show abstract

“…In addition, DRL-based methods have been employed for solving scheduling problems in manufacturing systems. In a hybrid flow shop scheduling problem, an agent allocates jobs from a given state that indicates whether the machine status is idle or busy or finished [51]. Among various shop scheduling problems, several researchers have investigated DRL-based methods for minimizing the makespan in job shop scheduling problems.…”

Section: Deep Reinforcement Learning For Solving Scheduling Problemsmentioning

confidence: 99%

Deep Reinforcement Learning for Minimizing Tardiness in Parallel Machine Scheduling With Sequence Dependent Family Setups

Paeng

Park

2021

IEEE Access

View full text Add to dashboard Cite

Parallel machine scheduling with sequence-dependent family setups has attracted much attention from academia and industry due to its practical applications. In a real-world manufacturing system, however, solving the scheduling problem becomes challenging since it is required to address urgent and frequent changes in demand and due-dates of products. To minimize the total tardiness of the scheduling problem, we propose a deep reinforcement learning (RL) based scheduling framework in which trained neural networks (NNs) are able to solve unseen scheduling problems without re-training even when such changes occur. Specifically, we propose state and action representations whose dimensions are independent of production requirements and due-dates of jobs while accommodating family setups. At the same time, an NN architecture with parameter sharing was utilized to improve the training efficiency. Extensive experiments demonstrate that the proposed method outperforms the recent metaheuristics, rule-based, and other RL-based methods in terms of total tardiness. Moreover, the computation time for obtaining a schedule by our framework is shorter than those of the metaheuristics and other RL-based methods.INDEX TERMS Deep reinforcement learning, unrelated parallel machine scheduling, sequence-dependent family setups, total tardiness objective, deep Q-network.

show abstract

Intelligent scheduling of discrete automated production line via deep reinforcement learning

Cited by 70 publications

References 15 publications

Deep reinforcement learning in production systems: a systematic literature review

Deep reinforcement learning in production systems: a systematic literature review

Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

Deep Reinforcement Learning for Minimizing Tardiness in Parallel Machine Scheduling With Sequence Dependent Family Setups

Contact Info

Product

Resources

About