Cost‐efficient monitoring of continuous‐time stochastic processes based on discrete observations

Abstract

Planning a cost‐efficient monitoring policy of stochastic processes arises from many industrial problems. We formulate a simple discrete‐time monitoring problem of continuous‐time stochastic processes with its applications to several industrial problems. A key in our model is a doubling trick of the variables, with which we can construct an algorithm to solve the problem. The cost‐efficient monitoring policy balancing between the observation cost and information loss is governed by an optimality equation of a fixed point type, which is solvable with an iterative algorithm based on the Feynman‐Kac formula. This is a new linkage between monitoring problems and mathematical sciences. We show regularity results of the optimization problem and present a numerical algorithm for its approximation. A problem having model ambiguity is presented as well. The presented model is applied to problems of environment, ecology, and energy, having qualitatively different target stochastic processes with each other.

Impulsive fishery resource transporting strategies based on an open‐ended stochastic growth model having a latent variable

“…PDE-constrained optimization problems based on FPEs have various applications; they are including but are not limited to bilinear optimal control [26], model predictive control [27], costefficient switching [28], pedestrian motion control [29], equilibrium firm dynamics [30], and impulsive mean field games [31]. The aforementioned examples of the FPEs in PDE-constrained optimization have a theoretical idealization that the decision-maker, the controller of the target stochastic system, can intervene the system at any time; however, it is often difficult to continuously intervene the target dynamics especially when the dynamics are biological ones in natural environment [32][33]. Problems in inland fisheries are no exception.…”

Robust Bellman State Prediction with Learning and Model Preferences

“…Snell envelope), by minimizing error between the state and its estimate across sampling intervals. Yoshioka et al (2021) use dynamic programming to derive the optimal monitoring schedule even for ambiguous processes by balancing observation cost and information loss. These contrast work reviewed by Heemels et al (2012) emphasizing schedules maximizing the next trigger-time, where performance targets bound a Lyapunov or similar function.…”

Markov decision processes with observation costs

“…More recently, Dyrssen and Ekström incorporated observation costs in hypothesis testing for the drift of a diffusion, and characterises the value function as the unique fixed point of an associated operator [12]. Other works concerning observation controls are motivated by specific applications: Winkelmann et al explored the optimal diagnosis and treatment scheduling for HIV-1 patients, based on the trade off between treatment cost against productivity loss across different countries [11,20,21]; Yoshioka et al focused on a variety of environmental management problems, including the modelling of replenishing sediment storage in rivers, monitoring algae population dynamics and biological growth of fishery resources [24,25,26,27]. The phenomenon of sporadically observing the state process is also modelled in mathematical finance under the term rational inattention, where portfolio adjustments are assumed to occur infrequently with a utility cost proportional to the users' assets [1,2,10].…”

“…Other existing works on the observation control model assume stationarity in the problem. This leads to a reward functional that only depends on the triplet (x, i, α) [12,21,24,25,26,27]. Whilst this formulation gives an overall lower dimensional problem, the setup assumes that the user is in possession of accurate and updated information of the state process at initialisation.…”