Utilizing Multi-Agent Deep Reinforcement Learning For Flexible Job Shop Scheduling Under Sustainable Viewpoints

Popper, Jens; Motsch, William; David, Alexander; Petzsche, Teresa; Ruskowski, Martin

doi:10.1109/iceccme52200.2021.9590925

Cited by 15 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering that a production scheduling task may be conceptualized as the environment within the framework of RL, an agent can acquire a policy of well-designed actions and states, and engage the extensive offline training through interaction with the environment. This innovative concept offers a fresh perspective on addressing scheduling challenges, particularly those characterized by uncertainty and dynamism, and necessitating stringent real-time constraints, as in the case of in a dynamic job shop scheduling problem [22,[43][44][45][46][47].…”

Section: Literature Reviewmentioning

confidence: 99%

“…Wang et al [37] discusses that the production scheduling process involves manufacturing of several types of items using a hybrid production pattern that utilizes Multi-Agent Deep Reinforcement Learning (MADRL) model. Popper et al [43] suggested that MADRL can be employed to optimize flexible production plants in a reactive manner, taking several criteria into account such as efficient and ecological target values. Du et al [61] utilized the DQN algorithm to address the flexible task shop scheduling problem (FJSP) in the presence of varying processing rates, setup time, idle time, and task transportation.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

A Review on Reinforcement Learning in Production Scheduling: An Inferential Perspective

Modrak,

Ranjitharamasamy,

Balamurugan

et al. 2024

Preprint

View full text Add to dashboard Cite

In this study, a comprehensive review on production scheduling based on reinforcement learning (RL) techniques using bibliometric analysis has been carried out. The aim of this work is, among other things, to point out the growing interest in this domain and to outline the influence of RL as a type of machine learning on production scheduling. To achieve this, the paper explores production scheduling using RL by investigating the descriptive metadata of all pertinent publications contained in the Web of Science database. The study focuses on a wide spectrum of publications spanning the years between 1998 and 2023. The findings and recommendations of this study can serve as new insights for future research endeavors in the realm of production scheduling using RL techniques.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

A Review on Reinforcement Learning in Production Scheduling: An Inferential Perspective

Modrak,

Ranjitharamasamy,

Balamurugan

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“… Alignment of the MAS4AI Solution with the RAMI4.0 Architecture (based on DIN SPEC 91345 , 2016 ; Alexopoulos et al, 2020 ; Popper et al, 2021 ). …”

Section: Key Concepts For Human-centered Smart Manufacturingmentioning

confidence: 99%

The MAS4AI framework for human-centered agile and smart manufacturing

Sidorenko,

Motsch,

van Bekkum

et al. 2023

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

Volatility and uncertainty of today's value chains along with the market's demands for low-batch customized products mandate production systems to become smarter and more resilient, dynamically and even autonomously adapting to both external and internal disturbances. Such resilient behavior can be partially enabled by highly interconnected Cyber-Physical Production Systems (CPPS) incorporating advanced Artificial Intelligence (AI) technologies. Multi-agent solutions can provide better planning and control, improving flexibility and responsiveness in production systems. Small modular parts can autonomously take intelligent decisions and react to local events. The main goal of decentralization and interconnectivity is to enable autonomous and cooperative decision-making. Nevertheless, a more efficient orchestration of various AI components and deeper human integration are required. In addition, global behaviors of coalitions of autonomous agents are not easily comprehensible by workers. Furthermore, it is challenging to implement an Industry 4.0 paradigm where a human should be in charge of decision-making and execution. This paper discusses a Multi-Agent System (MAS) where several software agents cooperate with smart workers to enable a dynamic and reconfigurable production paradigm. Asset Administration Shell (AAS) submodels hold smart workers' descriptions in machine-readable format, serving as an integration layer between various system's components. The self-description capability of the AAS supports the system's adaptability and self-configuration. The proposed concept supports the plug-and-produce functionality of the production modules and improves human-machine integration in the shared assembly tasks.

show abstract

“…Each machine is taken as a scheduler agent, which collects the scheduling states of all machines as input for training separately and executes the scheduling policy, respectively. Popper et al (2021) [32] proposed a distributed MARL scheduling method for the multiobjective optimization problem of minimizing energy consumption and delivery delay in the production process. The basic algorithm is solved by PPO, which regulates the joint behavior of each agent through a common reward function.…”

Section: Marl With Dtpmentioning

confidence: 99%

“…NMIR (normalized MIR), represented as Equation (32), reflects the deviation of the makespan solved under the dynamic environment from the best-known makespan.…”

Section: Generalization To Robot Breakdownsmentioning

confidence: 99%

Multi-Agent Reinforcement Learning for Extended Flexible Job Shop Scheduling

Peng,

Xiong,

Yang

et al. 2023

Machines

View full text Add to dashboard Cite

An extended flexible job scheduling problem is presented with characteristics of technology and path flexibility (dual flexibility), varied transportation time, and an uncertain environment. The scheduling can greatly increase efficiency and security in complex scenarios, e.g., distributed vehicle manufacturing, and multiple aircraft maintenance. However, optimizing the scheduling puts forward higher requirements on accuracy, real time, and generalization, while subject to the curse of dimension and usually incomplete information. Various coupling relations among operations, stations, and resources aggravate the problem. To deal with the above challenges, we propose a multi-agent reinforcement learning algorithm where the scheduling environment is modeled as a decentralized partially observable Markov decision process. Each job is regarded as an agent that decides the next triplet, i.e., operation, station, and employed resource. This paper is novel in addressing the flexible job shop scheduling problem with dual flexibility and varied transportation time in consideration and proposing a double Q-value mixing (DQMIX) optimization algorithm under a multi-agent reinforcement learning framework. The experiments of our case study show that the DQMIX algorithm outperforms existing multi-agent reinforcement learning algorithms in terms of solution accuracy, stability, and generalization. In addition, it achieves better solution quality for larger-scale cases than traditional intelligent optimization algorithms.

show abstract

Utilizing Multi-Agent Deep Reinforcement Learning For Flexible Job Shop Scheduling Under Sustainable Viewpoints

Cited by 15 publications

References 22 publications

A Review on Reinforcement Learning in Production Scheduling: An Inferential Perspective

A Review on Reinforcement Learning in Production Scheduling: An Inferential Perspective

The MAS4AI framework for human-centered agile and smart manufacturing

Multi-Agent Reinforcement Learning for Extended Flexible Job Shop Scheduling

Contact Info

Product

Resources

About