Accurate realtime full-body motion capture using a single depth camera

Abstract

Figure 1: Our system automatically and accurately reconstructs 3D skeletal poses in real time using monocular depth data obtained from a single camera. (top) reference image data; (bottom) the reconstructed poses overlaying depth data. AbstractWe present a fast, automatic method for accurately capturing fullbody motion data using a single depth camera. At the core of our system lies a realtime registration process that accurately reconstructs 3D human poses from single monocular depth images, even in the case of significant occlusions. The idea is to formulate the registration problem in a Maximum A Posteriori (MAP) framework and iteratively register a 3D articulated human body model with monocular depth cues via linear system solvers. We integrate depth data, silhouette information, full-body geometry, temporal pose priors, and occlusion reasoning into a unified MAP estimation framework. Our 3D tracking process, however, requires manual initialization and recovery from failures. We address this challenge by combining 3D tracking with 3D pose detection. This combination not only automates the whole process but also significantly improves the robustness and accuracy of the system. Our whole algorithm is highly parallel and is therefore easily implemented on a GPU. We demonstrate the power of our approach by capturing a wide range of human movements in real time and achieve state-ofthe-art accuracy in our comparison against alternative systems such as Kinect [2012].

Motion Capture Data Completion via Truncated Nuclear Norm Regularization

“…In this part, we present the completion results of six motion sequences (i.e. dance, walk, gymnastics, jump, score and boxing 3 ) from CMU mocap database 4 in the following experiments. We perform the simulations by fixing λ = (4), it is necessary to estimate the rank rank(X) of the incomplete motion data at first.…”

“…It is reasonable 3 The indices of the selected motions are 05 13, 12 02, 49 02, 13 13, 10 01 and 13 17, which consist of multiple types of action. 4 http://mocap.cs.cmu.edu/. because human motion is highly articulated so that most of the missing information can be revived through interpolation from neighboring markers.…”

“…However, the motion data captured is not complete even from professional systems, for example, some markers cannot be recorded due to occlusion, ambiguities or other factors. This problem is more severe for the mocap data acquired by Microsoft Kinect [4], [5]. Therefore, it is essential to complete the missing entries before further applications.…”

Real-Time Body Tracking with One Depth Camera and Inertial Sensors

“…Other real-time algorithms were proposed by e.g. [17] that use a body-part detector similar to [13] to augment a generative tracker. However, none of these hybrid approaches is able to give a meaningful pose hypothesis for non-visible body parts in case of occlusions.…”

“…[1,17] can track human skeletons in real-time from a single depth camera, as long as the body is mostly front-facing. However, even in frontal poses, tracking may fail due to complex self-occlusions, limbs close to the body, and other ambiguities.…”