Controlled Markov Fields with Finite State Space on Graphs

Abstract

Discrete time Markov chains with multidimensional state space are considered where the coordinates are locally interacting and develop synchronously. The interaction structure of the process is given by some general graph. Decision makers control the system's behaviour on the coordinate level using only local information. In the class of local strategies there exist deterministic stationary strategies which achieve minimal asymptotic average expected costs. If the cost structure is separable these strategies are even globally optimal.

Mathematical Modeling of Distributed Catastrophic and Terrorist Risks1

“…The number of publications in this field is rather insignificant up to date. Some results related to the general theory of controlled random fields and, in particular, Markov random fields [17][18][19][20][21] were obtained in [22][23][24][25], where optimal control problems for such fields are considered and the existence of optimal Markov stationary strategies for risk functions of rather general form is proved. It is obvious that these studies can be immediately applied to the risk assessment problems described above for rather general models of financial and actuarial mathematics.…”

Use of Steiner Problem in Solving Practical Problems of Road Construction

“…This makes graphs a very convenient form of data management for different tasks solving, using algorithms. Thus, the concept of the tree is actively used in computer science and programming (Emelichev, 1990;Chornei, Daduna & Knopov, 2005).…”

On Markov stochastic processes with local interaction for solving some applied problems

“…If the concept of the Markov property is introduced in such a manner that the distribution of losses at the point k depends only on losses in the neighborhood N k ( ), then, under very general conditions, a similar model is described by a Gibbs random field and the stated modeling problem is reduced to well-known methods of modelling Gibbs random fields. This is shown in more detail in [4,18,19]. In [4,19], problems of control of such fields are considered and the existence of optimal Markov stationary strategies for risk functions of general form is proved.…”

“…This is shown in more detail in [4,18,19]. In [4,19], problems of control of such fields are considered and the existence of optimal Markov stationary strategies for risk functions of general form is proved. It is obvious that the results of these investigations can be directly applied to the above-mentioned risk estimation problems for sufficiently general models of financial and actuarial mathematics.…”