Real-Time Posture Reconstruction for Microsoft Kinect

Abstract

The recent advancement of motion recognition using Microsoft Kinect stimulates many new ideas in motion capture and virtual reality applications. Utilizing a pattern recognition algorithm, Kinect can determine the positions of different body parts from the user. However, due to the use of a single-depth camera, recognition accuracy drops significantly when the parts are occluded. This hugely limits the usability of applications that involve interaction with external objects, such as sport training or exercising systems. The problem becomes more critical when Kinect incorrectly perceives body parts. This is because applications have limited information about the recognition correctness, and using those parts to synthesize body postures would result in serious visual artifacts. In this paper, we propose a new method to reconstruct valid movement from incomplete and noisy postures captured by Kinect. We first design a set of measurements that objectively evaluates the degree of reliability on each tracked body part. By incorporating the reliability estimation into a motion database query during run time, we obtain a set of similar postures that are kinematically valid. These postures are used to construct a latent space, which is known as the natural posture space in our system, with local principle component analysis. We finally apply frame-based optimization in the space to synthesize a new posture that closely resembles the true user posture while satisfying kinematic constraints. Experimental results show that our method can significantly improve the quality of the recognized posture under severely occluded environments, such as a person exercising with a basketball or moving in a small room.

Filtered pose graph for efficient kinect pose reconstruction

“…Results show that our method generated higher quality reconstructed poses than previous works, such as the original Kinect pose estimation [35] and the reconstruction method proposed in [37].…”

“…To improve the robustness of Kinect under complex environment, an effective method is to correct potential errors by reconstructing the unreliable part of the Kinect poses using prior knowledge of human movement [37]. The idea is to construct a database of accurately captured human poses as a prior to optimize the Kinect ones, so as to estimate the true pose performed by the user despite the errors returned by the Kinect.…”

“…Here, the key research problem is to identify the most relevant subset of poses in the database to accurately and efficiently reconstruct skeleton data. In particular, the brute force database search proposed by Shum et al [37] cannot ensure temporal continuity across frames. Thus, it reduces the relevance of the retrieved poses and therefore degrades the quality of the reconstruction.…”

“…In this paper, we introduce a data structure named Filtered Pose Graph to enhance the pose reconstruction process introduced by [37]. Using Filtered Pose Graph aims at ensuring continuity and relevance in the candidate selection, and consequently enhance the performance of the reconstruction.…”

Real-Time Body Gestures Recognition Using Training Set Constrained Reduction

“…As regards HMM-based solutions, the problem is usually faced up by using dimensionality reduction algorithms as the Principal Component Analysis (PCA) [20]. On the other hand, in DTW-based solutions, the reduction algorithms aims at reducing as much as possible the cardinality of the training set to very few and representative samples named prototypes (see for instance [21] and [22]).…”