Predictive Maintenance Decision Making Based on Reinforcement Learning in Multistage Production Systems

Abstract

Predictive maintenance has become increasingly prevalent in modern production systems that are challenged by high-mix low-volume production and short production life cycle. It is very helpful to prevent costly equipment failures, and reduce significant production loss caused by unscheduled machine breakdown. Although important, decision models for joint predictive maintenance and production in manufacturing systems have not been fully explored. Therefore, we propose a reinforcement learning based decision model, that brings together production system modeling and approximate dynamic programming. We start from the development of a state-based model by analyzing the dynamics of a multistage production system with predictive maintenance. It provides an approach to quantitatively evaluate the various disruptions as well as the maintenance decision's impact on production. Then a reinforcement learning method is proposed to explore optimal maintenance policies, that optimize the production and maintenance cost. To further improve the performance of the production system, machine stoppage bottlenecks are defined. An event-based indicator is proved to identify bottlenecks with production data. We test the proposed models in simulation case studies. The proposed predictive maintenance decision model is compared with three policies, which are state-based policy (SBP), time-based policy (TBP) and greedy policy (GP). The numerical studies show that the proposed decision model outperforms the policies, and it has the lowest system cost that is 9.68%, 39.07%, and 39.56% lower than SBP, TBP, and GP, respectively. In addition, the research shows that bottleneck identification and mitigation could help manufacturing systems to achieve more than 9.00% throughput improvement.

A Generalized Framework for Adopting Regression-Based Predictive Modeling in Manufacturing Environments

“…Similar work to the endeavor in [22] was reported in [10]. As a supposed improvement on the work carried out in [22], the exploration of optimal maintenance policies also represented by a Markov decision process is further augmented by considering machine stoppage bottlenecks on the shop or plant floors.…”

“…To do this, two approaches involving the improvement of the worst machine and a random machine, respectively, were considered with a focus on the reduction of the downtime duration. Even though machine stoppage bottlenecks were considered alongside numerical analysis involving comparisons with popular and widely accepted methods by manufacturers in [10] in addition to the work carried out in [22], the drawback of not thoroughly exploring the potential of predictive modeling to make a strong case for its adoption, given the manufacturing and/or production context investigated remains. Hence, this can still be viewed arguably as a relatively grey area of research interest considering the potential impact of the adoption of predictive modeling in manufacturing or production environments.…”

“…As hinted at above, manufacturing has always been data intensive. However, up until recent times, manufacturing data has not really been used to "automagically" improve or enhance the manufacturing processes that generated them in the first place [9,10]. In other words, other than process monitoring and regulation that are mostly achieved via traditional process visualization and process control using human-machine interfaces (HMIs), manufacturing data is still yet to be used extensively to support data-driven paradigms such as predictive modeling (usually takes the form of predictive maintenance when machine behavior and the life cycle is the context [11,12]) and ubiquitous monitoring and control (usually incorporated in the industrial internet of things (IIoT) where higher levels of connectivity, interoperability and robustness is the goal [13,14]).…”

Indirect Dynamic Negotiation in the Nash Demand Game

“…MDP allows to capture the underlying stochastics omnipresent in application domain and also allows to respect multiple DM criteria. Typical examples of using MDP framework include medical applications [49], predictive maintenance [50], power systems [51], more examples see [52].…”

Identification and alleviation of bottleneck in serial production line: a case study based on BOPP firm production

“…* Increase the production capacity of bottleneck equipment * Increase the number of bottleneck devices * Install buffers or increase capacity in front of bottleneck devices to reduce clogging rates In most of the past researches, various methods of definition, detection, and linkage have been proposed, but there is no definition or solution technique that is universally recognized and widely used in practical production, mainly due to the polymorphic, time-sensitive, and drifting nature of bottlenecks themselves and the diversity of bottlenecks in different application scenarios [5]. In this paper, we analyze the location of bottleneck generation and the attempts to mitigate bottlenecks based on mathematical analysis and computer simulation from the application point of view, in order to get the best conclusion on the application of theory to practice [6].…”