Deep Neural Network-Based Cooperative Visual Tracking Through Multiple Micro Aerial Vehicles

IEEE Robot. Autom. Lett.

Price

Ludwig

et al. 2019

Self Cite

We present a novel robotic front-end for autonomous aerial motion-capture (mocap) in outdoor environments. In previous work, we presented an approach for cooperative detection and tracking (CDT) of a subject using multiple micro-aerial vehicles (MAVs). However, it did not ensure optimal view-point configurations of the MAVs to minimize the uncertainty in the person's cooperatively tracked 3D position estimate. In this article, we introduce an active approach for CDT. In contrast to cooperatively tracking only the 3D positions of the person, the MAVs can actively compute optimal local motion plans, resulting in optimal view-point configurations, which minimize the uncertainty in the tracked estimate. We achieve this by decoupling the goal of active tracking into a quadratic objective and non-convex constraints corresponding to angular configurations of the MAVs w.r.t. the person. We derive this decoupling using Gaussian observation model assumptions within the CDT algorithm. We preserve convexity in optimization by embedding all the non-convex constraints, including those for dynamic obstacle avoidance, as external control inputs in the MPC dynamics. Multiple real robot experiments and comparisons involving 3 MAVs in several challenging scenarios are presented.

Section: Experiments and Resultsmentioning

confidence: 99%

“…2) Comparison of methods: For each of the following methods, we conducted 30 simulation runs with a 5 MAV formation: (i) SE 1: Approach of Price et al [1], (ii) SE 2: DQMPC (Tallamraju et al) [15], (iii) SE 3: Our approach, and (iv) SE 4. Our approach with all MAVs avoiding emulated virtual obstacles.…”

Section: B Simulation Experiments 1) Setupmentioning

confidence: 99%

Active Perception Based Formation Control for Multiple Aerial Vehicles

IEEE Robot. Autom. Lett.

Price

Ludwig

et al. 2019

Self Cite

“…The objective of the system of robots is to ensure that the centroid of the system bounding-box x B t reaches a desired destination position x B d t in the vicinity of the target position x T t . Our work is motivated by the application of simultaneous target tracking ( [17], [18]) and payload transportation. The key requirements in our target tracking scenario are, (i) to not lose track of the target, and, (ii) to ensure that the payload and the formation of robots avoid all the obstacles in their vicinity.…”

Section: B Motion Planning Overviewmentioning

confidence: 99%

Motion Planning for Multi-Mobile-Manipulator Payload Transport Systems

2019 IEEE 15th International Conference on Automation Science and Engineering (CASE)

Salunkhe

Rajappa

et al. 2019

Self Cite

In this paper, a kinematic motion planning algorithm for cooperative spatial payload manipulation is presented. A hierarchical approach is introduced to compute realtime collision-free motion plans for a formation of mobile manipulator robots. Initially, collision-free configurations of a deformable 2-D virtual bounding box are identified, over a planning horizon, to define a convex workspace for the entire system. Then, 3-D payload configurations whose projections lie within the defined convex workspace are computed. Finally, a convex decentralized model-predictive controller is formulated to plan collision-free trajectories for the formation of mobile manipulators. This approach facilitates real-time motion planning for the system and is scalable in the number of robots. The algorithm is validated in simulated dynamic environments. Simulation video: https://youtu.be/9EKj7RwRs_4.

“…In a laboratory setting MoCap is performed using a large number of precisely calibrated and high-resolution static cameras. To perform human MoCap in an outdoor setting or in an unstructured indoor environment, the use of multiple and autonomous micro aerial vehicles (MAVs) has recently gained attention [1], [2], [3], [4], [5]. Aerial MoCap of humans/animals facilitates several important applications, e.g., search and rescue using aerial vehicles, behavior estimation for endangered animal species, aerial cinematography and sports analysis.…”

Section: Introductionmentioning

confidence: 99%

AirCapRL: Autonomous Aerial Human Motion Capture Using Deep Reinforcement Learning

IEEE Robot. Autom. Lett.

Saini

Bonetto

et al. 2020

Self Cite

In this letter, we introduce a deep reinforcement learning (DRL) based multi-robot formation controller for the task of autonomous aerial human motion capture (MoCap). We focus on vision-based MoCap, where the objective is to estimate the trajectory of body pose and shape of a single moving person using multiple micro aerial vehicles. State-of-the-art solutions to this problem are based on classical control methods, which depend on hand-crafted system and observation models. Such models are difficult to derive and generalize across different systems. Moreover, the non-linearities and non-convexities of these models lead to sub-optimal controls. In our work, we formulate this problem as a sequential decision making task to achieve the vision-based motion capture objectives, and solve it using a deep neural network-based RL method. We leverage proximal policy optimization (PPO) to train a stochastic decentralized control policy for formation control. The neural network is trained in a parallelized setup in synthetic environments. We performed extensive simulation experiments to validate our approach. Finally, real-robot experiments demonstrate that our policies generalize to real world conditions.