Robust shortest path planning and semicontractive dynamic programming

Abstract

In this paper we consider shortest path problems in a directed graph where the transitions between nodes are subject to uncertainty. We use a minimax formulation, where the objective is to guarantee that a special destination state is reached with a minimum cost path under the worst possible instance of the uncertainty. Problems of this type arise, among others, in planning and pursuit-evasion contexts, and in model predictive control. Our analysis makes use of the recently developed theory of abstract semicontractive dynamic programming models. We investigate questions of existence and uniqueness of solution of the optimality equation, existence of optimal paths, and the validity of various algorithms patterned after the classical methods of value and policy iteration, as well as a Dijkstra-like algorithm for problems with nonnegative arc lengths. † Dimitri Bertsekas is with the Dept. of Electr. Engineering and Comp. Science, and the Laboratory for Information and Decision Systems, M.I.T., Cambridge, Mass., 02139.

A Dijkstra-Type Algorithm for Dynamic Games

“…For instance, if the game is the discretization of a control system where player a wants to reach a target in minimum time and b represents a disturbance, the lower value models the case that the controller can observe the disturbance and design his control strategy accordingly. Therefore, the upper value is more appropriate for a worst-case analysis of this problem, and this is in fact the choice in [23,25], and [11].…”

“…(13), the solution W of the discrete Isaacs equation (11) Proof Under the assumption (13), the discrete Isaacs equation (11) coincides with the onestep dynamic programming principle (5) satisfied by V − . On the other hand, from (5), one gets the general dynamic programming principle (4) by induction, and therefore, the equality W = V − .…”

“…iterating a finite number of times, where the set X n+1 is truncated to a finite set in case it is not finite, and then erasing all nodes that do not satisfy (11). The practical construction of an adapted grid is quite difficult in general, and it must exploit the geometry of the target T and of the vector fields.…”

“…An associated differential game can be chosen with dynamics The locally uniform convergence of the solution W of the discrete Isaacs equation (11) to the lower value function v − as h and the mesh size k of the grid X tend to zero was proved in [5] in the case v − is continuous, T has a Lipschitz boundary, and k/ h → 0, see also [17], the survey by Bardi et al [6], and the recent book by Falcone and Ferretti [19]. In the case of discontinuous v − , the convergence is weaker and the precise statements are more technical, see [4].…”

“…Some of such games related to ours are the planning problems against nature in [25], the reachability games studied in [1], and the robust shortest path problems of the very recent paper by Bertsekas [11]; all these references study also some numerical algorithms related to Dijkstra. Moreover, if the players move simultaneously and use random strategies, the game becomes a special case of the zero-sum stochastic games introduced in the seminal paper of Shapley [32].…”