Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines

Rodríguez, Marcelo Luis Ruiz; Kubler, Sylvain; Giorgio, Andrea de; Cordy, Maxime; Robert, Jérémy; Traon, Yves Le

doi:10.1016/j.rcim.2022.102406

Cited by 34 publications

(14 citation statements)

References 96 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the technical level of existing maintenance personnel is inadequate (Xue 2020). Therefore, it is especially important to improve the personal technical level, which will help to improve the effectiveness of maintenance management (Tang 2021, Wang 2020a, Ruiz Rodríguez et al 2022. Moreover, highway maintenance information is critical.…”

Section: Management Technologymentioning

confidence: 99%

A Hybrid Approach to Investigating Major Management Factors for Effective Highway Preventive Maintenance

Zhao,

Liu,

et al. 2024

Preprint

View full text Add to dashboard Cite

Highway preventive maintenance (HPM) can help reduce the negative environmental impacts of transportation infrastructure by prolonging the life of existing infrastructure, reducing the need for costly and resource-intensive repairs and reconstruction, and improving the energy efficiency of pavement infrastructure. However, many transportation agencies struggle with low HPM management capacity. This paper aims to enhance HPM management effectiveness by identifying and evaluating the major management factors that impact HPM. The study conducted a literature review and exploratory factor analysis (EFA) to identify the key HPM management factors. Social network analysis (SNA) was used to assess the importance of these factors, and a system dynamics (SD) model was developed to explore their influence laws. The research identified six dimensions of HPM management, including management system, management resource, management cognition, management decision, management technology, and external condition, along with 26 major management factors. The study found that key factors had a positive impact on HPM management, while hub factors were also critical. The study provides a comprehensive framework for identifying and evaluating the management factors that impact HPM, which can guide managers to develop effective HPM plans, improve the overall quality of highway maintenance, and form a sustainable transportation system.

show abstract

Section: Management Technologymentioning

confidence: 99%

A Hybrid Approach to Investigating Major Management Factors for Effective Highway Preventive Maintenance

Zhao,

Liu,

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The trial-and-error learning through the interaction with the environment and not requiring pre-collected data and prior expert knowledge allows RL algorithms to adapt to uncertain conditions, which is also discussed by Panzer and Bender (2022). Some applications can be found in manufacturing, for instance, in scheduling tasks as an example demonstrated by Dong, Xue, Xiao and Li (2020), maintenance as a case study researched by Rodríguez, Kubler, de Giorgio, Cordy, Robert and Le Traon (2022); Yousefi, Tsianikas and Coit (2022), process control described by the authors Spielberg, Tulsyan, Lawrence, Loewen and Gopaluni (2020), energy management example elaborated by Lu, Li, Li, Jiang and Ding (2020), assembly task mentioned by Tortorelli, Imran, Delli Priscoli and Liberati (2022), and robot manipulation that in detail has been discussed by Beltran-Hernandez, Petit, Ramirez-Alpizar and Harada (2020); Schoettler, Nair, Luo, Bahl, Ojea, Solowjow and Levine (2020).…”

Section: Related Work and Contributionmentioning

confidence: 99%

“…However, as fast as DRL is enhancing its capabilities to master its application in games, the gap between its real-world, safety-critical systems is becoming wider. There are comparatively few studies, where DRL has been implemented on safety-critical industrial cases, some of which are presented by Rodríguez et al (2022); Senthil and Sudhakara Pandian (2022); Spielberg, Gopaluni and Loewen (2017); Shin, Badgwell, Liu and Lee (2019).…”

Section: Related Work and Contributionmentioning

confidence: 99%

Interpretable Input-Output Hidden Markov Model-Based Deep Reinforcement Learning for the Predictive Maintenance of Turbofan Engines

Abbas

Chasparis

Kelleher

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

An open research question in deep reinforcement learning is how to focus the policy learning of key decisions within a sparse domain. This paper emphasizes combining the advantages of inputoutput hidden Markov models and reinforcement learning towards interpretable maintenance decisions. We propose a novel hierarchical-modeling methodology that, at a high level, detects and interprets the root cause of a failure as well as the health degradation of the turbofan engine, while, at a low level, it provides the optimal replacement policy. It outperforms the baseline performance of deep reinforcement learning methods applied directly to the raw data or when using a hidden Markov model without such a specialized hierarchy. It also provides comparable performance to prior work, however, with the additional benefit of interpretability.

show abstract

“…Lin et al (2019) [101] implemented a multi-agent DQN algorithm for a semiconductor manufacturing industry in order to cover the human-based decisions and reduce the complexity of the problem, resulting in enhanced performance. Through a similar approach, Ruiz R. et al (2022) [102] focus on the maintenance scheduling of several machines presenting up to ≈ 75% improvement in overall performance. Other studies combine those algorithms with IoT devices for smart resource allocation [168] or with other algorithms, such as Lamarckian local search for emergency scheduling activities [169].…”

Section: Schedulingmentioning

confidence: 99%

“…Moreover, thanks to the capacity of ANNs to create simple representations of complex inputs and functions, DRL algorithms can address complex tasks, maintaining adaptability and robustness [100]. Indeed, some applications can be found in manufacturing, for instance, in scheduling tasks [101,102] and robot manipulation [103,104].…”

mentioning

confidence: 99%

A Review of Deep Reinforcement Learning Approaches for Smart Manufacturing in Industry 4.0 and 5.0 Framework

et al. 2022

View full text Add to dashboard Cite

In this review, the industry’s current issues regarding intelligent manufacture are presented. This work presents the status and the potential for the I4.0 and I5.0’s revolutionary technologies. AI and, in particular, the DRL algorithms, which are a perfect response to the unpredictability and volatility of modern demand, are studied in detail. Through the introduction of RL concepts and the development of those with ANNs towards DRL, the potential and variety of these kinds of algorithms are highlighted. Moreover, because these algorithms are data based, their modification to meet the requirements of industry operations is also included. In addition, this review covers the inclusion of new concepts, such as digital twins, in response to an absent environment model and how it can improve the performance and application of DRL algorithms even more. This work highlights that DRL applicability is demonstrated across all manufacturing industry operations, outperforming conventional methodologies and, most notably, enhancing the manufacturing process’s resilience and adaptability. It is stated that there is still considerable work to be carried out in both academia and industry to fully leverage the promise of these disruptive tools, begin their deployment in industry, and take a step closer to the I5.0 industrial revolution.

show abstract

Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines

Cited by 34 publications

References 96 publications

A Hybrid Approach to Investigating Major Management Factors for Effective Highway Preventive Maintenance

A Hybrid Approach to Investigating Major Management Factors for Effective Highway Preventive Maintenance

Interpretable Input-Output Hidden Markov Model-Based Deep Reinforcement Learning for the Predictive Maintenance of Turbofan Engines

A Review of Deep Reinforcement Learning Approaches for Smart Manufacturing in Industry 4.0 and 5.0 Framework

Contact Info

Product

Resources

About