Reinforcement learning applications to machine scheduling problems: a comprehensive literature review

Kayhan, Behice Meltem; Yildiz, Gokalp

doi:10.1007/s10845-021-01847-3

Cited by 46 publications

(14 citation statements)

References 91 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In any case, some limitations that were revealed while conducting the study are an opportunity for new research lines and forthcoming works. Thus, we consider it necessary to develop more specific literature reviews oriented to RL and scheduling (Kayhan and Yildiz 2021). Furthermore, it would be desirable to extend RL and DRL approaches to new real-world strategical and tactical problems in the PPC areas of facility resource design specifically for supply chain design and facility and warehouse location problems, and also in the capacity planning and purchase and supply management areas to model and solve aggregate planning, lot-sizing and scheduling (Rummukainen and Nurminen 2019;Lang et al 2020;Zhang et al 2020), and even logistics problems (Rabe and Dross 2015).…”

Section: Discussionmentioning

confidence: 99%

Reinforcement learning applied to production planning and control

Esteso

Peidro

Díaz-Madroñero

2022

International Journal of Production Research

View full text Add to dashboard Cite

The objective of this paper is to examine the use and applications of reinforcement learning (RL) techniques in the production planning and control (PPC) field addressing the following PPC areas: facility resource planning, capacity planning, purchase and supply management, production scheduling and inventory management. The main RL characteristics, such as method, context, states, actions, reward and highlights, were analysed. The considered number of agents, applications and RL software tools, specifically, programming language, platforms, application programming interfaces and RL frameworks, among others, were identified, and 181 articles were sreviewed. The results showed that RL was applied mainly to production scheduling problems, followed by purchase and supply management. The most revised RL algorithms were model-free and single-agent and were applied to simplified PPC environments. Nevertheless, their results seem to be promising compared to traditional mathematical programming and heuristics/metaheuristics solution methods, and even more so when they incorporate uncertainty or non-linear properties. Finally, RL value-based approaches are the most widely used, specifically Q-learning and its variants and for deep RL, deep Q-networks. In recent years however, the most widely used approach has been the actor-critic method, such as the advantage actor critic, proximal policy optimisation, deep deterministic policy gradient and trust region policy optimisation.

show abstract

Section: Discussionmentioning

confidence: 99%

Reinforcement learning applied to production planning and control

Esteso

Peidro

Díaz-Madroñero

2022

International Journal of Production Research

View full text Add to dashboard Cite

show abstract

“…Meanwhile, we do not limit our scope to single problem styles, for example, manufacturing/machine scheduling or transportation [45–47], since there are indivisible associations between neural architectures or training algorithms in methods for the involved three COPs, which we believe are critical to stimulating more advanced methods in learning‐for‐optimisation community.…”

Section: Introductionmentioning

confidence: 99%

“…In contrast to existing surveys from the methodological perspective, we comprehensively present related works for each problem, which are not constrained to specific methods, for example, RL [41, 44–47], graph neural networks [38–40]. We hope it could grasp the whole picture for each problem, which readers would be interested in.…”

Section: Introductionmentioning

confidence: 99%

“…However, we regard production as one of the essential links of the manufacturing system with the other two major components, that is, packing and logistics, and provide a more systematic view of the whole system by covering each of the links via reviewing the state of the art research respectively. Although the detailed analysis of scheduling problems in the production system is given in refs [45,46], the literature introduced is relatively outdated compared with those in this paper. Furthermore, we focus on algorithms based on GNN and attention-based neural network, which are demonstrated as more suitable and promising for COPs in the manufacturing system, for example, for vehicle routing problem (VRP) [9].…”

mentioning

confidence: 99%

“…The corresponding literature on the methods for specific COPs are reviewed respectively in refs [32][33][34][35]. In addition, some surveys from the methodological perspective overview existing learning-based methods for COPs, for example, on generic usage of ML [1,36,37], graph neural networks [38][39][40], reinforcement learning (RL) [41][42][43][44][45][46][47].…”

mentioning

confidence: 99%

See 2 more Smart Citations

A review on learning to solve combinatorial optimisation problems in manufacturing

Zhang

et al. 2023

IET Collab Intel Manufact

View full text Add to dashboard Cite

An efficient manufacturing system is key to maintaining a healthy economy today. With the rapid development of science and technology and the progress of human society, the modern manufacturing system is becoming increasingly complex, posing new challenges to both academia and industry. Ever since the beginning of industrialisation, leaps in manufacturing technology have always accompanied technological breakthroughs from other fields, for example, mechanics, physics, and computational science. Recently, machine learning (ML) technology, one of the crucial subjects of artificial intelligence, has made remarkable progress in many areas. This study thoroughly reviews how ML, specifically deep (reinforcement) learning, motivates new ideas for addressing challenging problems in manufacturing systems. We collect the literature targeting three aspects: scheduling, packing, and routing, which correspond to three pivotal cooperative production links of today's manufacturing system, that is, production, packing, and logistics respectively. For each aspect, we first present and discuss the state-of-the-art research. Then we summarise and analyse the development trends and point out future research opportunities and challenges. K E Y W O R D Sbin packing, combinatorial optimisation, deep reinforcement learning, job shop scheduling, manufacturing systems, vehicle routing | INTRODUCTIONCombinatorial optimisation problems (COPs), as one important branch of mathematical optimisation, have practical applications in many fields, such as communication, transportation, manufacturing and aroused broad research in industrial engineering, computer science, and operations research. Due to the NP (non-deterministic polynomial-time) hardness, finding their optimal solutions is challenging. In specific, the discrete solution space in COPs renders the optimisation less efficient, without the guidance of gradient as in continuous optimisation. Meanwhile, the complexity of searching the (near-)optimal solution(s) among feasible solutions could exponentially increase as the problem scale grows. Classic methods, including exact algorithms and (meta-)heuristics, generally depend on massive expertise and tuning work to solve specific problems. They are Cong Zhang, Yaoxin Wu, and Yining Ma are equal contribution.

show abstract

Stochastic parallel machine scheduling using reinforcement learning

Julaiti

Park

Das

et al. 2022

J Adv Manuf & Process

View full text Add to dashboard Cite

In a high‐mix and low‐volume manufacturing facility, heterogeneous jobs introduce frequent reconfiguration of machines which increases the chance of unplanned machine breakdowns. As machines are often nonidentical and their performance degrades over time, it is critical to consider the heterogeneity and non‐stationarity of the machines during scheduling. We propose a reinforcement learning‐based framework with a novel sampling method to train the agent to schedule heterogeneous jobs on non‐stationary unreliable parallel machines to minimize weighted tardiness. The results indicate that the new sampling approach expedites the learning process and the resulting policy significantly outperforms static dispatching rules.

show abstract

Reinforcement learning applications to machine scheduling problems: a comprehensive literature review

Cited by 46 publications

References 91 publications

Reinforcement learning applied to production planning and control

Reinforcement learning applied to production planning and control

A review on learning to solve combinatorial optimisation problems in manufacturing

Stochastic parallel machine scheduling using reinforcement learning

Contact Info

Product

Resources

About