Optimization of costly measurements in stochastic decision processes

Abstract

Summary:Two kinds of problems regarding measurement optimization in stochastic decision processes, when measurements are costly or constrained not to exceed a given number, have been investigated in the last years: the first one refers to the optimum timing of observations on the state vector of the process, while the second refers to the convenience of buying information on the random actions exerted by a stochastic environment. In this paper the two problems are considered from a unified point of view. In other words, the decision maker has to determine the optimal observation policy, under the assumption that both state and random vectors are measurable. A solution based on the application of dynamic programming is discussed for a general class of multistage processes. Analytical results are then obtained for scalar linear systems with quadratic cost on state and control. In this case, an efficient algorithm to determine the optimal sequence of measurements is presented, on the base of which a sensitivity analysis with respect to the process parameters is also carried out. Zusammenfassung:In den letzten Jahren wurden zwei Typen yon Problemen untersucht, die sich auf die Optimierung yon Messungen bei stochastischen Entscheidungsprozessen beziehen, einmal der Fall, dal3 Messungen kostspielig sind, und zum anderen, dab deren Anzahl beschr~inkt ist. Der erste Fall bezieht sich auf die Gewinnung von Informationen fiber den stochastischen Sttirprozel3, der zweite Fall untersucht den optimalen Zeitabstand zwischen den Messungen des Zustands-(bzw. Ist-)-vektors. In diesem Beitrag wurden beide Probleme von einem gemeinsamen Gesichtspunkt aus betrachtet, d. h. der Entscheidungstr~iger soil die optimale Mel3politik bestimmen, unter der Annahme, dab Zustands-und St6rvektor meBbar sind. Mit Hilfe des Dynamischen Programmierens wird eine allgemeine LSsung des Problems hergeleitet. Sodann werden analytische Ergebnisse fiir den Fall linear-quadratischer Modelle angegeben. Schliel31ich wird ein wirksamer Algorithmus zur Bestimmung der optimalen Mel3folge entwickelt und darfiber hinaus eine Sensitivit~itsanalyse der Abh~ingig-keit yon den Prozel3parametern durchgeffihrt.

Optimization of moves and measurements in networks with stochastic costs

“…In stochastic optimization problems, quantities of this type are often called "value of information" or "expected value of perfect information" (EVPI). See, for example, Raiffa and Schlaifer [ 1961], Marschak and Radner [ 1972] and Puliafito and Zoppoli [ 1973] 5. Simplified Methods for Special Networks A serious shortcoming of the proposed method regards the number of states of the augmented graph that must be taken into consideration (actually, this is a typical shortcoming of dynamic programming).…”

Communication Problems in Decentralized Control Systems

“…It has been shown that the optimal control and measurement policies can be determined separately. The former j policy is obtained in the same way as in the classical optimal control theory, whereas the latter can be computed off-line by solving a certain nonlinear deterministic optimization j problem [1][2][3][4][5][6][7][8][9].…”