HPERL: 3D Human Pose Estimation from RGB and LiDAR

Abstract

In-the-wild human pose estimation has a huge potential for various fields, ranging from animation and action recognition to intention recognition and prediction for autonomous driving. The current state-of-the-art is focused only on RGB and RGB-D approaches for predicting the 3D human pose. However, not using precise LiDAR depth information limits the performance and leads to very inaccurate absolute pose estimation. With LiDAR sensors becoming more affordable and common on robots and autonomous vehicle setups, we propose an end-to-end architecture using RGB and LiDAR to predict the absolute 3D human pose with unprecedented precision. Additionally, we introduce a weakly-supervised approach to generate 3D predictions using 2D pose annotations from PedX [1]. This allows for many new opportunities in the field of 3D human pose estimation.

Neural Network Based Lidar Gesture Recognition for Realtime Robot Teleoperation

“…It is often performed using RGB and RGB-D data [13], [14]. In [15], the authors propose a multi-modal system using RGB imagary and lidar scans to obtain a precise 3D pose estimation. However, these methods require large CNNs and large amounts of data to achieve good results, since they use high-dimension inputs such as high-resolution images and dense point clouds to predict body poses.…”

“…5, and each one of them is mapped to a specific Fig. 5: Static (1-12) and dynamic (13)(14)(15)(16)(17)(18) gestures learned by the classifier (Images from [28]). command.…”