Distributed Real-Time Scheduling in Cloud Manufacturing by Deep Reinforcement Learning

Zhang, Lixiang; Yang, Chen; Yan, Yan; Hu, Yaoguang

doi:10.1109/tii.2022.3178410

Cited by 36 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yao et al [49] proposed a new mixed integer linear programming (MILP) model based on the improved disjunctive graph model, and verified its effectiveness in solving the job-shop scheduling problem of mobile robots through examples. Zhang et al [20] proposed a DRTS method based on deep reinforcement learning, which effectively solved the distributed real-time scheduling problem of job shop AGVs in cloud manufacturing mode. In general, this topic category revealed that under the guidance of lean logistics, production capacity and other constraints, Heuristic algorithm, machine learning, system dynamics, Witness, and other simulation modeling optimization methods are used to solve the problems of workshop production job scheduling, efficiency prediction, logistics AGV intelligent scheduling, combinatorial optimization logistics analysis, system layout design, and so on [50][51][52][53][54][55][56][57][58][59][60].…”

Section: Analysis Of Bibliometric Results Of Production Logistics In ...mentioning

confidence: 99%

“…In recent years, many scholars conducted research in the direction of production logistics process optimization, production logistics systems, and enabling technology applications. In production logistics process optimization, existing research focused on production logistics process optimization theories (e.g., lean logistics, green logistics), simulation modeling methods (e.g., Petri nets, heuristic algorithms) [17][18][19][20][21][22][23][24][25][26][27]. In terms of production logistics system research, existing studies focused on production equipment layout design, logistics scheduling, and process modeling (e.g., data flow diagram modeling, action diagram modeling) [14,28,29].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Production Logistics in Industry 3.X: Bibliometric Analysis, Frontier Case Study, and Future Directions

Zhang

et al. 2023

Systems

View full text Add to dashboard Cite

At present, the development of the global manufacturing industry is still in the transition stage from Industry 3.0 to Industry 4.0 (i.e., Industry 3.X), and the production logistics system is becoming more and more complex due to the individualization of customer demands and the high frequency of order changes. In order to systematically analyze the research status and dynamic evolution trend of production logistics in the Industry 3.X stage, this paper designed a Log-Likelihood ratio-based latent Dirichlet allocation (LLR-LDA) algorithm based on bibliometrics and knowledge graph technology, taking the literature of China National Knowledge Infrastructure and Web of Science database as the data source. In-depth bibliometric analysis of literature was carried out from research progress, hotspot evolution, and frontier trends. At the same time, taking the case of scientific research projects overcome by our research group as an example, it briefly introduced the synchronized decision-making framework of digital twin-enabled production logistics system. It is expected to broaden the research boundary of production logistics in the Industry 3.X stage, promote the development and progress of the industry, and provide valuable reference for steadily moving towards the Industry 4.0 stage.

show abstract

Section: Analysis Of Bibliometric Results Of Production Logistics In ...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Production Logistics in Industry 3.X: Bibliometric Analysis, Frontier Case Study, and Future Directions

Zhang

et al. 2023

Systems

View full text Add to dashboard Cite

show abstract

“…Since for each SFP type the same processing time is given, a constant o 1 ensures equidistant time steps for the agent-environment interactions. Otherwise, a more sophisticated approach such as a so called Semi-MDP would be needed to cope with varying production amounts and therefore non-equidistant time steps [26] [27]. We define o 1 to be 50 based on domain expertise.…”

Section: ) Action Spacementioning

confidence: 99%

Reinforcement Learning for Two-Stage Permutation Flow Shop Scheduling—A Real-World Application in Household Appliance Production

Müller,

Grumbach,

Kattenstroth

2024

IEEE Access

View full text Add to dashboard Cite

Solving production scheduling problems is a difficult and indispensable task for manufacturers with a push-oriented planning approach. In this study, we tackle a novel production scheduling problem from a household appliance production at the company Miele & Cie. KG, namely a two-stage permutation flow shop scheduling problem (PFSSP) with a finite buffer and sequence-dependent setup efforts. The objective is to minimize idle times and setup efforts in lexicographic order. In extensive and realistic data, the identification of exact solutions is not possible due to the combinatorial complexity. Therefore, we developed a reinforcement learning (RL) approach based on the Proximal Policy Optimization (PPO) algorithm that integrates domain knowledge through reward shaping, action masking, and curriculum learning to solve this PFSSP. Benchmarking of our approach with a state-of-the-art genetic algorithm (GA) showed significant superiority. Our work thus provides a successful example of the applicability of RL in real-world production planning, demonstrating not only its practical utility but also showing the technical and methodological integration of the agent with a discrete event simulation (DES). We also conducted experiments to investigate the impact of individual algorithmic elements and a hyperparameter of the reward function on the overall solution.

show abstract

“…Although CMfg has been around for several decades already, scheduling problems in CMfg have only gained significant interest in the last decade. The majority of published research in this area deals with dynamic scheduling problems using the multi-agent systems paradigm combined with artificial intelligence tools (Liu et al, 2019a;Rashidifar et al, 2022;Zhou et al, 2019;Wang et al, 2022c;Zhang et al, 2022;Liu et al, 2023). Two recent articles by Liu et al (2019a) and Rashidifar et al (2022) give a more complete review of this subject.…”

Section: Towards More Responsivenessmentioning

confidence: 99%

Flexible job shop scheduling problem under Industry 5.0: A survey on human reintegration, environmental consideration and resilience improvement

Destouet¹,

Tlahig²,

Bettayeb³

et al. 2023

Journal of Manufacturing Systems

View full text Add to dashboard Cite

Distributed Real-Time Scheduling in Cloud Manufacturing by Deep Reinforcement Learning

Cited by 36 publications

References 27 publications

Production Logistics in Industry 3.X: Bibliometric Analysis, Frontier Case Study, and Future Directions

Production Logistics in Industry 3.X: Bibliometric Analysis, Frontier Case Study, and Future Directions

Reinforcement Learning for Two-Stage Permutation Flow Shop Scheduling—A Real-World Application in Household Appliance Production

Flexible job shop scheduling problem under Industry 5.0: A survey on human reintegration, environmental consideration and resilience improvement

Contact Info

Product

Resources

About