Deep multi-agent reinforcement learning for multi-level preventive maintenance in manufacturing systems

Su, Jianyu; Huang, Jing; Adams, Stephen; Chang, Qing; Beling, Peter A.

doi:10.1016/j.eswa.2021.116323

Cited by 44 publications

(9 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Condition-based maintenance or predictive maintenance are typical scenarios where AI&ML can be leveraged (Black, Richmond, & Kolios, 2021;Carvalho et al, 2019). Engineering involves many sequential decision making; with rich information about engineering systems' status, reinforcement learning can be used to learn from simulations, experiments, routine operations, and generally experience for optimization, such as in mesh generation (Pan, Huang, Cheng, & Zeng, 2022), in manufacturing (Su, Huang, Adams, Chang, & Beling, 2022), for engineering design (Dworschak, Dietze, Wittmann, Schleich, & Wartzack, 2022). Machine learning can also be applied to even traditionally labour-intensive and time-consuming requirement elicitation process (Cheligeer et al, 2022;Mokammel et al, 2018).…”

Section: What Are the Emerging New Engineering Paradigms?mentioning

confidence: 99%

Digital engineering transformation with trustworthy AI towards industry 4.0: emerging paradigm shifts

Huang

2023

JID

View full text Add to dashboard Cite

Digital engineering transformation is a crucial process for the engineering paradigm shifts in the fourth industrial revolution (4IR), and artificial intelligence (AI) is a critical enabling technology in digital engineering transformation. This article discusses the following research questions: What are the fundamental changes in the 4IR? More specifically, what are the fundamental changes in engineering? What is digital engineering? What are the main uncertainties there? What is trustworthy AI? Why is it important today? What are emerging engineering paradigm shifts in the 4IR? What is the relationship between the data-intensive paradigm and digital engineering transformation? What should we do for digitalization? From investigating the pattern of industrial revolutions, this article argues that ubiquitous machine intelligence is the defining power brought by the 4IR. Digitalization is a condition to leverage ubiquitous machine intelligence. Digital engineering transformation towards Industry 4.0 has three essential building blocks: digitalization of engineering, leveraging ubiquitous machine intelligence, and building digital trust and security. The engineering design community at large is facing an excellent opportunity to bring the new capabilities of ubiquitous machine intelligence and trustworthy AI principles, as well as digital trust, together in various engineering systems design to ensure the trustworthiness of systems in Industry 4.0.

show abstract

Section: What Are the Emerging New Engineering Paradigms?mentioning

confidence: 99%

Digital engineering transformation with trustworthy AI towards industry 4.0: emerging paradigm shifts

Huang

2023

JID

View full text Add to dashboard Cite

show abstract

“…Math Prog. ML [27] 2022 GA [28] 2021 BH, VNS [29] 2021 GA, MC [30] 2021 GA [31] 2021 GA [32] 2021 GrA [33] 2021 GA [34] 2021 CEA [35] 2021 GrA [36] 2021 GA [37] 2021 BD [38] 2021 RHA [12] 2021 MP NN [39] 2021 RL [40] 2021 RL [41] 2020 GA [42] 2020 MVO [43] 2020 TS [44] 2020 NSGA-II [45] 2020 ANSGA-III [46] 2020 MA MIP [47] 2020 GA MIP [48] 2020 BOMP [49] 2020 BB [50] 2020 RL [51] 2020 RL [52] 2020 NN [53] 2019 GA [54] 2019 OTA [55] 2019 AC [56] 2019 NSGA-II [57] 2019 BIP [58] 2019 MILP [59] 2019 MINLP [60] 2019 RL [61] 2019 RL [62] 2019 RL [10] 2018 GA [63] 2018 SA, GA [64] 2018 SA [65] 2017 NSGA-II [66] 2017 MA [67] 2017 MILP [68] 2017 RL [69] 2016 NSGA-II [70] 2016 LPT MILP…”

Section: Yearmentioning

confidence: 99%

“…The RL policy outperforms CM and PM policies with respect to completed jobs and evidenced how the agents learn to execute maintenance closer to failure times with a low buffer volume. Su et al [40] extended Huang et al work [50,61] of single agents to multi agents for PM purposes on a serial production line with multi-stage machines. Their main contribution is to demonstrate convergence towards better policies and to solve scalability issues by the joint action space of the single agent approaches.…”

Section: Yearmentioning

confidence: 99%

Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines

Rodríguez

Kubler²,

Giorgio³

et al. 2022

Robotics and Computer-Integrated Manufacturing

View full text Add to dashboard Cite

“…This work incorporates a CNN-LSTM-based architecture for the DQN, while the impacts of agents are neglected. Su et al (2022) presented a MARL approach using value decomposition actor–critic (VDAC) to enable physical machines (in a serial production line that requires multiple levels of machine decisions) to learn local maintenance policies in a distributed and cooperative manner. The proposed solution is formulated as a DEC-POMDP problem, and CTDE was used to provide the solution.…”

Section: Applicationsmentioning

confidence: 99%

A review of the applications of multi-agent reinforcement learning in smart factories

Bahrpeyma

Reichelt

2022

Front. Robot. AI

View full text Add to dashboard Cite

The smart factory is at the heart of Industry 4.0 and is the new paradigm for establishing advanced manufacturing systems and realizing modern manufacturing objectives such as mass customization, automation, efficiency, and self-organization all at once. Such manufacturing systems, however, are characterized by dynamic and complex environments where a large number of decisions should be made for smart components such as production machines and the material handling system in a real-time and optimal manner. AI offers key intelligent control approaches in order to realize efficiency, agility, and automation all at once. One of the most challenging problems faced in this regard is uncertainty, meaning that due to the dynamic nature of the smart manufacturing environments, sudden seen or unseen events occur that should be handled in real-time. Due to the complexity and high-dimensionality of smart factories, it is not possible to predict all the possible events or prepare appropriate scenarios to respond. Reinforcement learning is an AI technique that provides the intelligent control processes needed to deal with such uncertainties. Due to the distributed nature of smart factories and the presence of multiple decision-making components, multi-agent reinforcement learning (MARL) should be incorporated instead of single-agent reinforcement learning (SARL), which, due to the complexities involved in the development process, has attracted less attention. In this research, we will review the literature on the applications of MARL to tasks within a smart factory and then demonstrate a mapping connecting smart factory attributes to the equivalent MARL features, based on which we suggest MARL to be one of the most effective approaches for implementing the control mechanism for smart factories.

show abstract

Deep multi-agent reinforcement learning for multi-level preventive maintenance in manufacturing systems

Cited by 44 publications

References 40 publications

Digital engineering transformation with trustworthy AI towards industry 4.0: emerging paradigm shifts

Digital engineering transformation with trustworthy AI towards industry 4.0: emerging paradigm shifts

Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines

A review of the applications of multi-agent reinforcement learning in smart factories

Contact Info

Product

Resources

About