AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation

Saini, Nitin; Bonetto, Elia; Price, Eric; Ahmad, Aamir; Black, Michael J.

doi:10.1109/lra.2022.3145494

Cited by 23 publications

(16 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These are limited in the number of subjects, clothing variety, and scenarios. Synthetic data is being used to solve these problems [26], [27]. However, such data does not include either the full camera's state, IMU readings, scene depth, LiDAR data, or offers the possibility to easily extend it after the experiment has been recorded (e.g.…”

Section: B Dynamic Contentmentioning

confidence: 99%

“…with additional cameras or sensors), thus is generally unusable for any robotics application. Indeed, synthetic datasets are usually developed by stitching people over image backgrounds [27], statically placing them in some limited environment [26] and often recorded with static monocular cameras that take single pictures [26]. Furthermore, many of those are generated without any clothing information [28].…”

Section: B Dynamic Contentmentioning

confidence: 99%

See 1 more Smart Citation

GRADE: Generating Realistic Animated Dynamic Environments for Robotics Research

Bonetto¹,

Xu²,

Ahmad³

2023

Preprint

View full text Add to dashboard Cite

Simulation engines like Gazebo, Unity and Webots are widely adopted in robotics. However, they lack either full simulation control, ROS integration, realistic physics, or photorealism. Recently, synthetic data generation and realistic rendering advanced tasks like target tracking and human pose estimation. However, when focusing on vision applications, there is usually a lack of information like sensor measurements (e.g. IMU, LiDAR, joint state), or time continuity. On the other hand, simulations for most robotics applications are obtained in (semi)static environments, with specific sensor settings and low visual fidelity. In this work, we present a solution to these issues with a fully customizable framework for generating realistic animated dynamic environments (GRADE) for robotics research. The data produced can be post-processed, e.g. to add noise, and easily expanded with new information using the tools that we provide. To demonstrate GRADE, we use it to generate an indoor dynamic environment dataset and then compare different SLAM algorithms on the produced sequences. By doing that, we show how current research over-relies on well-known benchmarks and fails to generalize. Furthermore, our tests with YOLO and Mask R-CNN provide evidence that our data can improve training performance and generalize to real sequences. Finally, we show GRADE's flexibility by using it for indoor active SLAM, with diverse environment sources, and in a multi-robot scenario. In doing that, we employ different control, asset placement, and simulation techniques. The code, results, implementation details, and generated data are provided as open-source. The main project page is https://eliabntt.github.io/grade-rr while the accompanying video can be found at https://youtu.be/cmywCSD-9TU.

show abstract

Section: B Dynamic Contentmentioning

confidence: 99%

Section: B Dynamic Contentmentioning

confidence: 99%

GRADE: Generating Realistic Animated Dynamic Environments for Robotics Research

Bonetto¹,

Xu²,

Ahmad³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The initial SMPL position and orientation in the first frame act as the pivot for all the optimizing parameters. Since optimizing all the parameters together is more likely to get stuck in a local minimum, we follow previous work [23] and do optimization in three phases. In all the optimization stages, we minimize the same loss function, which is a weighted combination of multiple loss terms.…”

Section: Camera and Human Pose Estimationmentioning

confidence: 99%

“…E COS and E CPS are the camera motion smoothing terms for camera orientation and position, respectively. Following the previous work [23], we use a simple L2 loss on the positions and the 6D representation of the camera orientations. They are given as…”

Section: Camera and Human Pose Estimationmentioning

confidence: 99%

See 1 more Smart Citation

SmartMocap: Joint Estimation of Human and Camera Motion using Uncalibrated RGB Cameras

Saini¹,

Huang²,

Black³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Markerless human motion capture (mocap) from multiple RGB cameras is a widely studied problem. Existing methods either need calibrated cameras or calibrate them relative to a static camera, which acts as the reference frame for the mocap system. The calibration step has to be done a priori for every capture session, which is a tedious process, and re-calibration is required whenever cameras are intentionally or accidentally moved. In this paper, we propose a mocap method which uses multiple static and moving extrinsically uncalibrated RGB cameras. The key components of our method are as follows. First, since the cameras and the subject can move freely, we select the ground plane as a common reference to represent both the body and the camera motions unlike existing methods which represent bodies in the camera coordinate. Second, we learn a probability distribution of short human motion sequences (∼1sec) relative to the ground plane and leverage it to disambiguate between the camera and human motion. Third, we use this distribution as a motion prior in a novel multi-stage optimization approach to fit the SMPL human body model and the camera poses to the human body keypoints on the images. Finally, we show that our method can work on a variety of datasets ranging from aerial cameras to smartphones. It also gives more accurate results compared to the state-of-the-art on the task of monocular human mocap with a static camera. Our code is available for research purposes on https://github.com/robot-perception-group/SmartMocap.

show abstract

Semantic Synthesis of Pedestrian Locomotion

Priisalu

Păduraru

Pirinen

et al. 2021

Computer Vision – ACCV 2020

View full text Add to dashboard Cite

It is difficult to perform 3D reconstruction from on-vehicle gathered video due to the large forward motion of the vehicle. Even object detection and human sensing models perform significantly worse on onboard videos when compared to standard benchmarks because objects often appear far away from the camera compared to the standard object detection benchmarks, image quality is often decreased by motion blur and occlusions occur often. This has led to the popularisation of traffic data-specific benchmarks. Recently Light Detection And Ranging (LiDAR) sensors have become popular to directly estimate depths without the need to perform 3D reconstructions. However, LiDAR-based methods still lack in articulated human detection at a distance when compared to image-based methods. We hypothesize that benchmarks targeted at articulated human sensing from LiDAR data could bring about increased research in human sensing and prediction in traffic and could lead to improved traffic safety for pedestrians.

show abstract

AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation

Cited by 23 publications

References 29 publications

GRADE: Generating Realistic Animated Dynamic Environments for Robotics Research

GRADE: Generating Realistic Animated Dynamic Environments for Robotics Research

SmartMocap: Joint Estimation of Human and Camera Motion using Uncalibrated RGB Cameras

Semantic Synthesis of Pedestrian Locomotion

Contact Info

Product

Resources

About