A note on the costly surveillance of a stochastic system

Abstract

We consider the costly surveillance of a stochastic system with a finite state space and a finite number of actions in each state. There is a positive cost of observing the system and the system earns at a rate depending on the state of the system and the action taken. A policy for controlling such a system specifies the action to be taken and the time to the next observation, both possibly random and depending on the past history of the system. A form of the long range average income is the criterion for comparing different policies. If R A denotes the class of policies for which the times between successive observations of the system are random variables with cumulative distribution functions on [0, A], A < m, we show that there exists a nonrandomized stationary policy that is optimal in RA. Furthermore, for sufficiently large A, this optimal policy is independent of A.

Dynamic Forecasting and Control Algorithms of Glaucoma Progression for Clinician Decision Support

“…In non-Markovian surveillance and inspection models (see Antelman and Savage 1965, Nakagawa and Yasui 1980, Kander and Raviv 1974, Chitgopekar 1974, the state space is still one-dimensional and the observations are assumed to be perfect. Papers that consider noisy or uncertain observations include Savage (1964), Noonan and Fain (1962), Rosenfield (1976), Eckles (1968), Ohnishi et al (1986b).…”