Distributed Differential Dynamic Programming Architectures for Large-Scale Multi-Agent Control

Abstract

In this paper, we propose two novel decentralized optimization frameworks for multi-agent nonlinear optimal control problems in robotics. The aim of this work is to suggest architectures that inherit the computational efficiency and scalability of Differential Dynamic Programming (DDP) and the distributed nature of the Alternating Direction Method of Multipliers (ADMM). In this direction, two frameworks are introduced. The first one called Nested Distributed DDP (ND-DDP), is a three-level architecture which employs ADMM for enforcing a consensus between all agents, an Augmented Lagrangian layer for satisfying local constraints and DDP as each agent's optimizer. In the second approach, both consensus and local constraints are handled with ADMM, yielding a two-level architecture called Merged Distributed DDP (MD-DDP), which further reduces computational complexity. Both frameworks are fully decentralized since all computations are parallelizable among the agents and only local communication is necessary. Simulation results that scale up to thousands of vehicles and hundreds of drones verify the effectiveness of the methods. Superior scalability to large-scale systems against centralized DDP and centralized/decentralized sequential quadratic programming is also illustrated. Finally, hardware experiments on a multi-robot platform demonstrate the applicability of the proposed algorithms, while highlighting the importance of optimizing for feedback policies to increase robustness against uncertainty. A video including all results is available here https://youtu.be/tluvENcWldw.

Decentralized iLQR for Cooperative Trajectory Planning of Connected Autonomous Vehicles via Dual Consensus ADMM

“…This is followed by dual consensus ADMM [33], [34], which extends the consensus ADMM to an optimization problem whose Lagrange dual problem is a consensus optimization problem. The recent applications of ADMM to cooperative trajectory planning are exemplified by [35]- [38]. Particularly in [35]- [37], ADMM is applied to separate the independent constraints from the coupling constraints of the multi-agent system, such that the former ones can be handled in a parallel manner, yielding a partially parallel solution.…”

“…Particularly in [35]- [37], ADMM is applied to separate the independent constraints from the coupling constraints of the multi-agent system, such that the former ones can be handled in a parallel manner, yielding a partially parallel solution. Meanwhile, a fully decentralized optimization framework is proposed in [38] to solve multirobot cooperative trajectory planning problems over a large scale. However, its performance is still far away from being real-time, even with a relatively small number of agents.…”

“…(40) In particular, r [1],τ represents the piece of r [1] from row number 2N (N − 1)τ to row number 2N (N − 1)(τ + 1) − 1, ri [2] represents the piece of r i [2] from row number (i − 1)T m to row number iT m − 1, and ri [2],τ represents the piece of ri [2] from row number τ m to (τ + 1)m − 1, inclusively. Plug ( 16), (39), and (40) back into (38), and we can show that δX i,k+1 is the optimizer of the following minimization problem…”