Sequential testing of a Wiener process with costly observations

Optim Control Appl Methods

Tsujimura

Hamagami

et al. 2020

Summary A recent river environmental restoration problem is approached from a standpoint of stochastic control of hybrid regime‐switching diffusion processes with discrete and costly observations. This setting harmonizes with real problems because continuously obtaining environmental and ecological information is difficult, and is often costly. The main problem is to decide when and how much of the sediment should be supplied into a river environment to effectively suppress bloom of benthic algae. The interventions are allowed only at observation times. Finding the optimal river restoration policy ultimately reduces to solving an optimality equation in an unconventional form due to the observation cost and discrete observations. We show its unique solvability and present a connection with degenerate parabolic partial differential equations, with which we can construct an effective algorithm for its approximation. An uncertainty‐averse optimization problem is also considered as an advanced problem. Coefficients and parameter values are identified from experimental and observation results to numerically compute the optimal restoration policy of an existing river environment with and without uncertainty.

“…Set a natural filtration generated by the observation process:

ℱ_{t}^{\overset{τ}{true}} = σ \{{((),,,, τ_{j}, α_{τ_{j}}, X_{τ_{j}}, Y_{τ_{j}})}_{0 \leq j \leq k}, k = \sup \{j : τ_{j} \leq t\}\} .

Following Dyrssen and Ekström 48 and Yoshioka and Tsujimura, 46 we only consider sequences

\overset{true^}{τ} = {({}, τ_{k})}_{k = 0, 1, 2, \dots}

such that each τ k is

ℱ^{\overset{τ}{true}}

‐predictable. The collection of all the admissible policies is denoted as

𝒜

, which is defined as follows.Definition The admissible set

𝒜

contains the pair

(\overset{τ}{true}, \overset{η}{true})

satisfying all the conditions below : τ 0 = 0, τ k + 1 ≥ τ k + ω ( k = 0, 1, 2, …), η 0 = 0, η k ∈ [0, 1] ( k = 1, 2, 3, …), τ k is

ℱ_{τ_{k - 1} +}^{\overset{τ}{true}}

‐ measurable ( k = 1, 2, 3, …), and η k is

ℱ_{τ_{k}}^{\overset{τ}{true}}

‐ measurable ( k = 1, 2, 3, …).…”

Section: Mathematical Modelmentioning

confidence: 99%

“…The starting point of this strategy is the dynamic programming principle result 46,48,49 formally leading to the optimality equation

Φ (i, x, y) = \underset{t \geq ω}{normalinf} {normalE}_{i, x, y} [e^{- δ italict} ℳ Φ (α_{t}, X_{t}, Y_{t}) + \int_{([),, 0 +, t)} e^{- δ italics} f (X_{s}) d s] for (i, x, y) \in Ω .

The intervention operator

ℳ

is defined as.

ℳ g (i, x, y) = \underset{η \in [0, 1]}{normalinf} (d + c (η) + g (i, x, \min \{y + η, 1\})) for (i, x, y) \in Ω

for generic sufficiently regular g .…”

Section: Mathematical Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A hybrid stochastic river environmental restoration modeling with discrete and costly observations

Optim Control Appl Methods

Tsujimura

Hamagami

et al. 2020

“…This is addressed based on Yoshioka and Tsujimura 35 who considered impulsive observation and harvesting of biological population. Their formulation is based on Dyrssen and Ekström 36 and Wang 37 who established discrete and costly observations models of the optimal stopping type. A target process to be monitored and a performance index to be optimized through monitoring activities comprise our mathematical model.…”

Section: Introductionmentioning

confidence: 99%

“…A main difference between the present and the previous problems 35‐37 is that the former aims at collecting information through the monitoring (namely, observations) without any interventions, while the latter aims at optimizing interventions (harvesting or terminating) through observations. Our problem is simpler in this sense.…”

Section: Introductionmentioning

confidence: 99%

Cost‐efficient monitoring of continuous‐time stochastic processes based on discrete observations

Appl Stoch Models Bus & Ind

Yaegashi

Tsujimura

et al. 2020

Planning a cost‐efficient monitoring policy of stochastic processes arises from many industrial problems. We formulate a simple discrete‐time monitoring problem of continuous‐time stochastic processes with its applications to several industrial problems. A key in our model is a doubling trick of the variables, with which we can construct an algorithm to solve the problem. The cost‐efficient monitoring policy balancing between the observation cost and information loss is governed by an optimality equation of a fixed point type, which is solvable with an iterative algorithm based on the Feynman‐Kac formula. This is a new linkage between monitoring problems and mathematical sciences. We show regularity results of the optimization problem and present a numerical algorithm for its approximation. A problem having model ambiguity is presented as well. The presented model is applied to problems of environment, ecology, and energy, having qualitatively different target stochastic processes with each other.

Stochastic impulse control of nonsmooth dynamics with partial observation and execution delay: Application to an environmental restoration problem

Optim Control Appl Methods

Yaegashi²

2021

Nonsmooth dynamics driven by stochastic disturbance arise in a wide variety of engineering problems. Impulsive interventions are often employed to control stochastic systems; however, the modeling and analysis subject to execution delay have been less explored. In addition, continuously receiving information of the dynamics is not always possible. In this article, with an application to an environmental restoration problem, a continuous-time stochastic impulse control problem subject to execution delay under discrete and random observations is newly formulated and analyzed. The dynamics have a nonsmooth coefficient modulated by a Markov chain, and eventually attain an undesirable state like a depletion due to the nonsmoothness. The goal of the control problem is to find the most cost-efficient policy that can prevent the dynamics from attaining the undesirable state. We demonstrate that finding the optimal policy reduces to solving a nonstandard system of degenerate elliptic equations, the Hamilton-Jacobi-Bellman equation (HJBE), which is rigorously and analytically verified in a simplified case. The associated Fokker-Planck equation (FPE) for the controlled dynamics is derived and solved explicitly as well. The model is finally applied to numerical computation of a recent river environmental restoration problem. The HJBE and FPE are successfully computed, and the optimal policy and the probability density functions are numerically obtained.The impacts of execution delay are discussed to deeper analyze the model.