Markerless Outdoor Human Motion Capture Using Multiple Autonomous Micro Aerial Vehicles

Saini, Nitin; Price, Eric; Tallamraju, Rahul; Enficiaud, Raffi; Ludwig, Roman; Martinovic, Igor; Ahmad, Aamir; Black, Michael J.

doi:10.1109/iccv.2019.00091

Cited by 34 publications

(34 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their approach works only indoors and the initial registration step can take a long time similar to other marker based method setups. In one of our previous works, [10], we introduced a vision-based (monocular RGB) markerless motion capture method using multiple aerial robots in outdoor scenarios. The pose and shape of the subject and the pose of the cameras are jointly estimated and optimized in [10].…”

Section: Related Workmentioning

confidence: 99%

“…In a laboratory setting MoCap is performed using a large number of precisely calibrated and high-resolution static cameras. To perform human MoCap in an outdoor setting or in an unstructured indoor environment, the use of multiple and autonomous micro aerial vehicles (MAVs) has recently gained attention [1], [2], [3], [4], [5]. Aerial MoCap of humans/animals facilitates several important applications, e.g., search and rescue using aerial vehicles, behavior estimation for endangered animal species, aerial cinematography and sports analysis.…”

Section: Introductionmentioning

confidence: 99%

“…The system's robotic front-end [2] must ensure that the subject i) is accurately and continuously followed by all aerial robots, and ii) is within the field of view (FOV) of the cameras of all robots. The back-end of the system estimates the 3D pose and shape of the subject, using the images and other data acquired by the front-end [1]. The front-end poses a formation control problem for multiple MAVs.…”

Section: Introductionmentioning

confidence: 99%

“…In existing solutions [1], [2], [3] the front and back end are developed independently -the formation control algorithms of the existing aerial MoCap front ends assume that the person should be centered in every MAV's camera image and she/he should be within a threshold distance to each MAV. These assumptions are intuitive and important.…”

Section: Introductionmentioning

confidence: 99%

“…We showcase the performance of our method in several simulation experiments. We evaluate the quality of the generated robot trajectories using the pose and shape estimation algorithms in [8], [9] and [1]. Additionally, we compare our new approach with the state-of-the-art modelbased controller from [2].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

AirCapRL: Autonomous Aerial Human Motion Capture Using Deep Reinforcement Learning

Tallamraju

Saini

Bonetto

et al. 2020

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

In this letter, we introduce a deep reinforcement learning (DRL) based multi-robot formation controller for the task of autonomous aerial human motion capture (MoCap). We focus on vision-based MoCap, where the objective is to estimate the trajectory of body pose and shape of a single moving person using multiple micro aerial vehicles. State-of-the-art solutions to this problem are based on classical control methods, which depend on hand-crafted system and observation models. Such models are difficult to derive and generalize across different systems. Moreover, the non-linearities and non-convexities of these models lead to sub-optimal controls. In our work, we formulate this problem as a sequential decision making task to achieve the vision-based motion capture objectives, and solve it using a deep neural network-based RL method. We leverage proximal policy optimization (PPO) to train a stochastic decentralized control policy for formation control. The neural network is trained in a parallelized setup in synthetic environments. We performed extensive simulation experiments to validate our approach. Finally, real-robot experiments demonstrate that our policies generalize to real world conditions.

show abstract

Section: Related Workmentioning

confidence: 99%