Reinforcement learning-based flow management of gas turbine parts under stochastic failures

Compare, Michele; Bellani, Luca; Cobelli, Enrico; Zio, Enrico

doi:10.1007/s00170-018-2690-6

Cited by 18 publications

(11 citation statements)

References 40 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application of model-free reinforcement learning (RL) to maintenance has been explored recently. Examples include on-policy RL (e.g., SARSA algorithm [21], [22]) proposed for a petroleum industry production system [23], for opportunistic maintenance of a fleet of military trucks [24], for minimizing the forced outage in gas turbine maintenance [25], for minimizing the average inventory level and This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ This article has been accepted for publication in a future issue of this journal, but has not been fully edited.…”

Section: (Deep) Reinforcement Learning For Maintenance Planningmentioning

confidence: 99%

Combinatorial Q-Learning for Condition-Based Infrastructure Maintenance

Tanimoto

2021

IEEE Access

View full text Add to dashboard Cite

Infrastructure maintenance planning is a large-scale optimization problem of planning when and on which components to carry out maintenance so as to keep the whole infrastructure in good condition with minimal maintenance cost. Recent advances in condition monitoring techniques have enabled timely maintenance in response to the condition of each part regardless of age. In addition to the condition, the spatial structure is also important for cost-efficiency in infrastructure maintenance since traveling costs and/or setup costs can be saved by simultaneous maintenance of neighboring components, which is called economic dependency. This optimization problem naively has a high computational complexity of O(2 nH ), where n is the number of components and H is the planning horizon, and the predictive modeling of degradation is also a big issue. To solve this problem efficiently at scale, our proposed method utilizes two kinds of dynamic programming for temporal and spatial scalability and consequently enjoys O(n) complexity at each time step. For temporal scalability, we utilize a direct modeling approach for the action value of maintenance instead of modeling degradation, namely, Q-learning. For spatial scalability, we exploit locality in economic dependency by means of a reasonable approximation of the Q-function. A typical baseline approach is to divide the whole infrastructure into fixed groups of neighboring components beforehand and determine if maintenance should be performed for all the components in each group at each time step. In contrast, our scalable method enables fully combinatorial optimization for each component at each time step. We demonstrate the advantage of our method in a simulated environment, and the resulting maintenance history intuitively illustrates the benefit of our dynamic grouping approach. We also show that our method has a kind of interpretability in the optimization at each time step.

show abstract

Section: (Deep) Reinforcement Learning For Maintenance Planningmentioning

confidence: 99%

Combinatorial Q-Learning for Condition-Based Infrastructure Maintenance

Tanimoto

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The downtimes of the WTs due to PM and CM actions, Π P M and Π CM , respectively, are random variables obeying probability density functions f ΠPM and f ΠCM , respectively. The downtime of a PM action is expected to be shorter than that of a CM, as all the maintenance logistic support issues have already been addressed (Compare et al (2018)). The costs of the preventive and corrective maintenance actions on each WT are U P M and U CM , respectively.…”

Section: Problem Statementmentioning

confidence: 99%

Deep Reinforcement Learning for Optimizing Operation and Maintenance of Energy Systems Equipped with PHM Capabilities

Pinciroli¹,

Baraldi²,

Ballabio³

et al. 2020

Proceedings of the 30th European Safety and Reliability Conference and 15th Probabilistic Safety Assessment and Management Conf

Self Cite

View full text Add to dashboard Cite

The Life Cycle Cost (LCC) of energy systems including Renewable Energy Sources (RES) strongly depends on the Operation and Maintenance (O&M) costs. Nowadays, many components of these energy systems are equipped with Prognostics & Health Management (PHM) capabilities, for estimating their current and future health states. This information is intended to be used for the optimization of O&M. It is an ambitious and challenging objective as the uncertain information brought by PHM must be combined with other factors influencing O&M, such as the limited availability of maintenance crews, the variability of energy demand and production, the long-time horizons of energy systems. In this work, we formalize the O&M optimization of RES-based energy systems equipped with PHM as a sequential decision problem over a long-time horizon and we solve it by Deep Reinforcement Learning (DRL). The proposed methodology is applied to a small wind farm. Strengths and weaknesses are analyzed by means of a comparison with state-of-the-art O&M policies.

show abstract

“…Barde et al [ 10 ] took multiple independent components in a military truck into account and found the optimal maintenance time for each component to minimize the system downtime by Monte Carlo RL. Compare et al [ 11 ] developed a framework by a Sarsa algorithm to find the best part flow management strategy of gas turbines consisting of repair and purchase actions in order to minimize the cost. In a more recent study, Bellani et al [ 12 ] proposed a Prognostics and Health Management (PHM) framework based on sequential decision-making and RL that, compared with the current practice of PHM, allows consideration of the maintenance and operation decisions’ influence on predicting the degradation.…”

Section: Introductionmentioning

confidence: 99%

Condition-Based Maintenance with Reinforcement Learning for Dry Gas Pipeline Subject to Internal Corrosion

Mahmoodzadeh

Droguett

et al. 2020

Sensors

View full text Add to dashboard Cite

Gas pipeline systems are one of the largest energy infrastructures in the world and are known to be very efficient and reliable. However, this does not mean they are prone to no risk. Corrosion is a significant problem in gas pipelines that imposes large risks such as ruptures and leakage to the environment and the pipeline system. Therefore, various maintenance actions are performed routinely to ensure the integrity of the pipelines. The costs of the corrosion-related maintenance actions are a significant portion of the pipeline’s operation and maintenance costs, and minimizing this large cost is a highly compelling subject that has been addressed by many studies. In this paper, we investigate the benefits of applying reinforcement learning (RL) techniques to the corrosion-related maintenance management of dry gas pipelines. We first address the rising need for a simulated testbed by proposing a test bench that models corrosion degradation while interacting with the maintenance decision-maker within the RL environment. Second, we propose a condition-based maintenance management approach that leverages a data-driven RL decision-making methodology. An RL maintenance scheduler is applied to the proposed test bench, and the results show that applying the proposed condition-based maintenance management technique can reduce up to 58% of the maintenance costs compared to a periodic maintenance policy while securing pipeline reliability.

show abstract

Reinforcement learning-based flow management of gas turbine parts under stochastic failures

Cited by 18 publications

References 40 publications

Combinatorial Q-Learning for Condition-Based Infrastructure Maintenance

Combinatorial Q-Learning for Condition-Based Infrastructure Maintenance

Deep Reinforcement Learning for Optimizing Operation and Maintenance of Energy Systems Equipped with PHM Capabilities

Condition-Based Maintenance with Reinforcement Learning for Dry Gas Pipeline Subject to Internal Corrosion

Contact Info

Product

Resources

About