A Dijkstra-Type Algorithm for Dynamic Games

Abstract

We study zero-sum dynamic games with deterministic transitions and alternating moves of the players. Player 1 aims at reaching a terminal set and minimizing a possibly discounted running and final cost. We propose and analyze an algorithm that computes the value function of these games extending Dijkstra's algorithm for shortest paths on graphs. We also show the connection of these games with numerical schemes for differential games of pursuit-evasion type, if the grid is adapted to the dynamical system. Under suitable conditions, we prove the convergence of the value of the discrete game to the value of the differential game as the step of approximation tends to zero.

Robust shortest path planning and semicontractive dynamic programming

“…Our Dijkstra‐like algorithm is patterned after the Dijkstra‐like stochastic shortest path algorithm, but requires less restrictive conditions because an optimal proper policy has the essential character of a consistently improving policy when the arc lengths are nonnegative. As noted earlier, related Dijkstra‐like algorithms were proposed by (without the type of convergence analysis that we give here), and by (under the assumption that all arc lengths are strictly positive, and with an abbreviated convergence argument). We will assume the following: Assumption …”

“…Problems of this kind have been studied extensively from different points of view (see e.g., Refs. 1,6,7,10,12,13,47,52,58,59,68,80). For our shortest path formulation to be applicable to such a problem, the pursuers and the evaders must have perfect information about each others' positions, and the Cartesian product of their positions (the state of the system) must be restricted to the finite set of nodes of a given graph, with known transition costs (i.e., a "terrain map" that is known a priori).…”

“…Sequential minimax problems have also been studied in the context of sequential games (starting with the paper [Sha53], and followed up by many other works, including the books [BaB91], [FiV96], and the references given there). Sequential games that involve shortest paths are particularly relevant; see the works [PaB99], [GrJ08], [Yu11], [BaL15]. An important difference with some of the works on sequential games is that in our minimax formulation, we assume that the antagonistic opponent knows the decision and corresponding subset of successor nodes chosen at each node.…”

“…Thus in our problem, it would make a difference if the decisions at each node were made with advance knowledge of the opponent's choice ("min-max" is typically not equal to "max-min" in our context). Generally shortest path games admit a simpler analysis when the arc lengths are assumed nonnegative (as is done for example in the recent works [GrJ08], [BaL15]), when the problem inherits the structure of negative DP (see [Str66], or the texts [Put94], [Ber12]) or abstract monotone increasing abstract DP models (see [Ber77], [BeS78], [Ber13]). However, our formulation and line of analysis is based on the recently introduced abstract semicontractive DP model of [Ber13], and allows negative as well as nonnegative arc lengths.…”

Numerical event-based ISS controller design via a dynamic game approach

“…In contrast to approaches like, e.g., finite elements which require fine discretizations [5], the set oriented discretization is particularly suitable for the quantized event-based problem formulation due to its ability to rigorously handle large quantization regions by representing them as boxes or cells in the set oriented discretization. The resulting game on a hypergraph can then be solved using a Dijkstra-type algorithm [11,26] (see also [3] for a recent extension).…”