A distributed approach solving partially flexible job-shop scheduling problem with a Q-learning effect

Bouazza, Wassim; Sallez, Yves; Beldjilali, Bouziane

doi:10.1016/j.ifacol.2017.08.2354

Cited by 59 publications

(28 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Wang et al [19] encapsulated the Q learning algorithm in Agents to train a processing machine on a single-machine scheduling problem; they used three DRs as candidate actions of the algorithm to minimize the average delay time. Bouazzad et al [20] improved the Q-table in the traditional Q learning algorithm by storing machine selection probabilities and the probability of specific rules, which allocate the most suitable processing machines and the processing sequence of the jobs in a dynamic jobshop. Shiue et al [21] used RL algorithms with Multiple Dispatching Rule (MDR) mechanisms and offline learning modules to maintain the Knowledge Base (KB) of a Real-Time Scheduling System (RTSS) that changes with the workshop environment.…”

Section: Literature Reviewmentioning

confidence: 99%

Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

Zhao¹,

Wang

Tan

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Jobshop scheduling is a classic instance in the field of production scheduling. Solving and optimizing the scheduling process of the jobshop can greatly reduce the production cost of the workshop and improve the processing efficiency, thereby improving the market competitiveness of the manufacturing industry. A reasonable scheduling algorithm in jobshop can reduce manufacturing costs, shorten product delivery time, enhance customer service quality, and improve the overall performance of the manufacturing system. The jobshop scheduling problem is transformed into a reinforcement learning problem based on the Markov decision process. The performance of the adaptive scheduling algorithm in a dynamic manufacturing environment is improved based on the Deep Q Network (DQN). In the proposed scheduling algorithm, five state features of continuous value ranges are designed for input to a Deep Neural Network (DNN), as well as ten well-known heuristic dispatching rules are selected as the action set of the DQN. In the proposed scheduling algorithm, the target network and the prediction network are used to train the parameters. The DQN uses a "softmax" function as an action selection strategy during the training process, where it selects the dispatching rules with the greatest action value as the execution action. It resolves the problem whereby the sub-optimal action value is greater than the Q value of the optimal action in the early learning stage. Futhermore, the non-optimal action is selected with a greater probability in the later learning stage. Ten typical jobshop test instances called "LA" are used as a simulation object and operated in Python. The simulation results confirm that the proposed scheduling algorithm based on DQN has better performance and universality than a single dispatching rule or traditional Q learning algorithm.INDEX TERMS dynamic scheduling, deep Q network, deep reinforcement learning, dispatching rules, job shop scheduling.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

Zhao¹,

Wang

Tan

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Other studies develop a heuristics model and algorithm to optimize and support scheduling flexible job-shop systems [26][27][28][29][30]. In 2017, a heterarchical approach based on intelligent products was analyzed by Bouazza et al [31]. Dolgui with Ivanov and Sokolov [32] developed a model of job shop scheduling in a customized manufacturing process.…”

Section: Literature Review On Fjss and Smart Manufacturingmentioning

confidence: 99%

A Dispatching-Fuzzy AHP-TOPSIS Model for Scheduling Flexible Job-Shop Systems in Industry 4.0 Context

et al. 2021

View full text Add to dashboard Cite

Scheduling flexible job-shop systems (FJSS) has become a major challenge for different smart factories due to the high complexity involved in NP-hard problems and the constant need to satisfy customers in real time. A key aspect to be addressed in this particular aim is the adoption of a multi-criteria approach incorporating the current dynamics of smart FJSS. Thus, this paper proposes an integrated and enhanced method of a dispatching algorithm based on fuzzy AHP (FAHP) and TOPSIS. Initially, the two first steps of the dispatching algorithm (identification of eligible operations and machine selection) were implemented. The FAHP and TOPSIS methods were then integrated to underpin the multi-criteria operation selection process. In particular, FAHP was used to calculate the criteria weights under uncertainty, and TOPSIS was later applied to rank the eligible operations. As the fourth step of dispatching the algorithm, the operation with the highest priority was scheduled together with its initial and final time. A case study from the smart apparel industry was employed to validate the effectiveness of the proposed approach. The results evidenced that our approach outperformed the current company’s scheduling method by a median lateness of 3.86 days while prioritizing high-throughput products for earlier delivery.

show abstract

“…Later, a deep Q-learning agent was trained to select the appropriate dispatching rules. In a distributed way, [34] used a Q-learning algorithm associated with Intelligent Products (IP) which collected data to pinpoint the current scheduling context, and then determined the most suitable machine selection rule and dispatching rule in a dynamic flexible job shop scheduling problem with new job insertion. The authors of [35] proposed a multi-agent system containing machine, buffer, state and job agents for dynamic job shop scheduling to minimize earliness and tardiness punishment.…”

Section: Job Shop Scheduling Using Artificial Intelligencementioning

confidence: 99%

A Q-Learning Rescheduling Approach to the Flexible Job Shop Problem Combining Energy and Productivity Objectives

2021

View full text Add to dashboard Cite

The flexible job shop problem (FJSP) has been studied in recent decades due to its dynamic and uncertain nature. Responding to a system’s perturbation in an intelligent way and with minimum energy consumption variation is an important matter. Fortunately, thanks to the development of artificial intelligence and machine learning, a lot of researchers are using these new techniques to solve the rescheduling problem in a flexible job shop. Reinforcement learning, which is a popular approach in artificial intelligence, is often used in rescheduling. This article presents a Q-learning rescheduling approach to the flexible job shop problem combining energy and productivity objectives in a context of machine failure. First, a genetic algorithm was adopted to generate the initial predictive schedule, and then rescheduling strategies were developed to handle machine failures. As the system should be capable of reacting quickly to unexpected events, a multi-objective Q-learning algorithm is proposed and trained to select the optimal rescheduling methods that minimize the makespan and the energy consumption variation at the same time. This approach was conducted on benchmark instances to evaluate its performance.

show abstract

A distributed approach solving partially flexible job-shop scheduling problem with a Q-learning effect

Cited by 59 publications

References 17 publications

Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network

A Dispatching-Fuzzy AHP-TOPSIS Model for Scheduling Flexible Job-Shop Systems in Industry 4.0 Context

A Q-Learning Rescheduling Approach to the Flexible Job Shop Problem Combining Energy and Productivity Objectives

Contact Info

Product

Resources

About