A reinforcement learning approach to optimal part flow management for gas turbine maintenance

Abstract

We consider the maintenance process of gas turbines used in the Oil and Gas industry: the capital parts are first removed from the gas turbines and replaced by parts of the same type taken from the warehouse; then, they are repaired at the workshop and returned to the warehouse for use in future maintenance events. Experience-based rules are used to manage the flow of the parts for a profitable gas turbine operation. In this article, we formalize the part flow management as a sequential decision problem and propose reinforcement learning for its solution. An application to a scaled-down case study derived from real industrial practice shows that reinforcement learning can find policies outperforming those based on experience-based rules.

Reinforcement learning-based flow management of gas turbine parts under stochastic failures

“…Currently, the management of the part flow is dealt with experiencebased rules, such as the most residual cycles (MRC) one: the removed parts are always repaired and the part with the largest residual life among those available at the warehouse are installed on the GT; a new part is purchased only when the warehouse is empty. Although MRC ensures at the smallest failure probability, nonetheless, we have shown in [25] that MRC does not necessarily yield optimal policies on a finite time horizon in which the sequence of MSs is a priori known.…”

“…To formalize the part flow management in a stochastic environment, the model proposed in [25] must be modified to allow decisions to be taken upon FOs, which occur at time instants different from those initially scheduled for the MSs. In fact, any failure event requires a re-scheduling of the maintenance activities, which entails a variability in both the number of events over the GT plant operation horizon and their timing and sequence.…”

“…In this work, we extend the modeling and optimization framework developed in [25] to account for part failure stochastic processes and FOs, which change the prescheduled sequence of MSs. In particular, we formalize the PFM problem as a SDP in a stochastic environment and propose the use of Reinforcement Learning (RL, [18,26,27]) for its solution.…”

“…The problem formulation and solution framework proposed in this paper is applied to the same case study as that of [25], although here, we take into account the failure of the parts and the FOs. In the case study, it turns out that also when considering the part failures, the solution given by the MRC rule is not optimal, being outperformed by the policy found by RL.…”

“…In the case study, it turns out that also when considering the part failures, the solution given by the MRC rule is not optimal, being outperformed by the policy found by RL. Moreover, we compare the optimal policy provided by our RL algorithm in case there are no FOs with that presented in [25].…”