An optimal inspection and replacement policy under incomplete state information

Abstract:We examine the problem of adaptively scheduling perfect observations and preventive replacements for a multi-state, Markovian deterioration system with silent failures such that total expected discounted cost is minimized. We model this problem as a partially observed Markov decision process and show that the structural properties of the optimal policy hold for certain non-extreme sample paths.

show abstract

“…Before presenting Proposition 3, we state several wellknown (cf. Rosenfield [6], White [9], Ohnishi [4]) results via Propositions 1 and 2.…”

Section: Preliminary Definitions and Resultsmentioning

confidence: 90%

Structured maintenance policies on interior sample paths

Maillart

Zheltova

2007

Naval Research Logistics

View full text Add to dashboard Cite

show abstract

“…For example, Givon and Grosfeld-Nir [12] develop a model for the replacement of TV shows that can also be applied to the replacement of binary-state systems, and their Proposition 1 is akin to our Corollary 1, but they assume that c r = 0 and p 22 = 1. Structural results by Ohnishi et al [30] on the optimal inspection and replacement policy for multi-state systems imply a threshold structure for the optimal policy in the binary-state case, but their results do not characterize this threshold structure in the same detail as in Corollaries 1 and 2, and their assumptions require c d ≥ c r and p 22 ≥ p 12 . The special case we consider here also is a special case of the model of Dada and Marcellus [7], who distinguish between "routine maintenance" and "learning maintenance."…”

Section: Special Case Without Sensor Deteriorationmentioning

confidence: 96%

“…There exist models in which costless, imperfect observations of the state of the system are made at every decision epoch (White , Ghasemi et al , Grosfeld‐Nir ) and models in which an observation of the state of the system is only made at a decision epoch if it is decided to conduct a costly inspection (Ross , Rosenfield , Maillart and Zheltova ). The combination of costless, imperfect observations and costly observations is considered in White and Ohnishi et al . In all papers listed above, it is assumed that the probabilistic relation between the state of the system and the observation that is obtained, which in POMDP models is described by an (action‐dependent) observation matrix, is stationary over time.…”

Section: Introductionmentioning

confidence: 99%

Optimal maintenance policies for a safety‐critical system and its deteriorating sensor

Oosterom

Maillart

Kharoufeh

2017

Naval Research Logistics

View full text Add to dashboard Cite

We consider the integrated problem of optimally maintaining an imperfect, deteriorating sensor and the safety‐critical system it monitors. The sensor's costless observations of the binary state of the system become less informative over time. A costly full inspection may be conducted to perfectly discern the state of the system, after which the system is replaced if it is in the out‐of‐control state. In addition, a full inspection provides the opportunity to replace the sensor. We formulate the problem of adaptively scheduling full inspections and sensor replacements using a partially observable Markov decision process (POMDP) model. The objective is to minimize the total expected discounted costs associated with system operation, full inspection, system replacement, and sensor replacement. We show that the optimal policy has a threshold structure and demonstrate the value of coordinating system and sensor maintenance via numerical examples. © 2017 Wiley Periodicals, Inc. Naval Research Logistics 64: 399–417, 2017

show abstract

“…Furthermore, a policy exists 11,13 that achieves this minimum cost. This optimal policy is stationary; (δ * ) ∞ , where the control function δ * selects the minimizing alternative in the relation (6).…”

Section: The Pomdp Modelmentioning

confidence: 99%

“…[1][2][3][4][5][6][7][8][9][10] Unfortunately, the computational burden associated with solving POMDPs is overwhelming, precluding their application to problems of practical size. [11][12][13] Computing the exact optimal finite or infinite horizon solution of a POMDP is generally extremely computationally intractable. For this reason many researchers consider approximation scheme for planning in partially stochastic domains.…”

Section: Introductionmentioning

confidence: 99%

Partially Observable Markov Decision Processes and Periodic Policies With Applications

Goulionis

Stengos

2011

Int. J. Info. Tech. Dec. Mak.

View full text Add to dashboard Cite

This paper treats the infinite horizon discounted cost control problem for partially observable Markov decision processes. Sondik studied the class of finitely transient policies and showed that their value functions over an infinite time horizon are piecewise linear (p.w.l) and can be computed exactly by solving a system of linear equations. However, the condition for finite transience is stronger than is needed to ensure p.w.l. value functions. In this paper, we introduce alternatively the class of periodic policies whose value functions turn out to be also p.w.l. Moreover, we examine a more general condition than finite transience and periodicity that ensures p.w.l. value functions. We implement these ideas in a replacement problem under Markovian deterioration, investigate for periodic policies and give numerical examples.

show abstract

An optimal inspection and replacement policy under incomplete state information

Cited by 53 publications

References 6 publications

Structured maintenance policies on interior sample paths

Structured maintenance policies on interior sample paths

Optimal maintenance policies for a safety‐critical system and its deteriorating sensor

Partially Observable Markov Decision Processes and Periodic Policies With Applications

Contact Info

Product

Resources

About