Planning for large-scale multiagent problems via hierarchical decomposition with applications to UAV health management

Abstract

This paper introduces a novel hierarchical decomposition approach for solving Multiagent Markov Decision Processes (MMDPs) by exploiting coupling relationships in the reward function. MMDP is a natural framework for solving stochastic multi-stage multiagent decision-making problems, such as optimizing mission performance of Unmanned Aerial Vehicles (UAVs) with stochastic health dynamics. However, computing the optimal solutions is often intractable because the state-action spaces scale exponentially with the number of agents. Approximate solution techniques do exist, but they typically rely on extensive domain knowledge. This paper presents the Hierarchically Decomposed MMDP (HD-MMDP) algorithm, which autonomously identifies different degrees of coupling in the reward function and decomposes the MMDP into a hierarchy of smaller MDPs that can be solved separately. Solutions to the smaller MDPs are embedded in an autonomously constructed tree structure to generate an approximate solution to the original problem. Simulation results show HD-MMDP obtains more cumulative reward than that of the existing algorithm for a ten-agent Persistent Search and Track (PST) mission, which is a cooperative multi-UAV mission with more than 10 19 states, stochastic fuel consumption model, and health progression model.

Decoupled multiagent path planning via incremental sequential convex programming

“…The distinctions between coupled and decoupled formulations, and between the two ways of adding constraints, are explained in following subsections. This section establishes that (1) decoupled formulations have significantly better computational tractability than the coupled formulations, (2) decoupled formulations of existing SCP approaches often fail to find feasible solutions in convex and non-convex environments, and (3) decoupled formulations of iSCP address the underlying problem leading to (2). To clarify further discussion, we classify various approaches of SCP in Table I.…”

“…Decoupled SCP (dec-SCP) 2 , shown in Alg. 1, presents improved computational tractability and enables decentralized, asynchronous implementation (explained in Sec.…”

“…Many applications of multiagent systems require coordination of mobile agents navigating in complex environments. For instance, researchers have investigated using fleets of unmanned aerial and ground vehicles for forest fire monitoring [1], persistent surveillance [2], warehouse inventory management [3], and expressive artistic performance [4]. These scenarios require computing collision free trajectories connecting initial and final positions for every agent in the fleet.…”

Adaptive Terminal Sliding Mode Control for Attitude and Position Tracking Control of Quadrotor UAVs in the Existence of External Disturbance

“…Generally speaking, UAVs have attracted much attention during recent years and they are able to carry out various particular atmospheric skills, for instance, exploration of moving objects [1], [2], accumulating of traffic data [3], large-scale systems [4], examination of power transition lines [5], organizing military operations' supplies [6] and supplying first-aid kit in natural disasters [7]. On the whole, unmanned aerial vehicles can be applied in military and civilian operations [8][9][10][11].…”

Measurable Augmented Reality for Prototyping Cyberphysical Systems: A Robotics Platform to Aid the Hardware Prototyping and Performance Testing of Algorithms

“…An example application is intruder monitoring using a team of quadrotors [26]. In this problem, a team of autonomous ground vehicles attempts to reach a goal location in a discretized world while a competing team of quadrotors attempts to push or "herd" them away.…”