Potential iLQR: A Potential-Minimizing Controller for Planning Multi-Agent Interactive Trajectories

Abstract

Many robotic applications involve interactions between multiple agents where an agent's decisions affect the behavior of other agents. Such behaviors can be captured by the equilibria of differential games which provide an expressive framework for modeling the agents' mutual influence. However, finding the equilibria of differential games is in general challenging as it involves solving a set of coupled optimal control problems. In this work, we propose to leverage the special structure of multi-agent interactions to generate interactive trajectories by simply solving a single optimal control problem, namely, the optimal control problem associated with minimizing the potential function of the differential game. Our key insight is that for a certain class of multi-agent interactions, the underlying differential game is indeed a potential differential game for which equilibria can be found by solving a single optimal control problem. We introduce such an optimal control problem and build on single-agent trajectory optimization methods to develop a computationally tractable and scalable algorithm for planning multi-agent interactive trajectories. We will demonstrate the performance of our algorithm in simulation and show that our algorithm outperforms the state-of-the-art game solvers. To further show the real-time capabilities of our algorithm, we will demonstrate the application of our proposed algorithm in a set of experiments involving interactive trajectories for two quadcopters.

GTP-SLAM: Game-Theoretic Priors for Simultaneous Localization and Mapping in Multi-Agent Scenarios

“…In particular, when cost terms that couple different players are all symmetric, there exists a single optimal control problem whose solution gives Nash equilibrium strategies of the interacting players. Recent literature indicates that iterative methods which exploit this symmetry structure in potential games can be more efficient than those that do not [23]. Our approach draws inspiration from this observation: we recast the multi-agent SLAM problem under study as a potential game, and perform IBR in a manner that preserves its potential structure.…”

“…In contrast, our work uses an iterative best response (IBR) scheme, in which each player takes a turn solving for their optimal strategy while assuming all other players' strategies are fixed [21][22][23][24]. By replacing the dynamic game with a sequence of optimal control problems, the computational burden of solving for a local Nash equilibrium strategy is substantially reduced.…”

“…Our work also draws inspiration from the potential games literature, which exploits the cost structure of multi-player interactions in certain robotics applications [23,24]. In particular, when cost terms that couple different players are all symmetric, there exists a single optimal control problem whose solution gives Nash equilibrium strategies of the interacting players.…”

“…Definition 2: (Potential Dynamic Game, [23], [26]) An N -player, K-stage general-sum dynamic game Γ is called a potential game if there exists an optimal control problem, defined over all players' controls (u 1 , • • • , u N ), whose solutions are Nash equilibria of the game Γ.…”

“…This connection to potential games is critical, as it suggests a locally-convergent solution method for GTP-SLAM problems given in Section IV-C (Corollary 1). The following results are based upon established concepts in the literature [23,24,26]; here, we illustrate their pertinence to the noncooperative SLAM problem.…”