Posture reconstruction using Kinect with a probabilistic model

Abstract

Recent work has shown that depth image based 3D posture estimation hardware such as Kinect has made interactive applications more popular. However, it is still challenging to accurately recognize postures from a single depth camera due to the inherently noisy data derived from depth images and self-occluding action performed by the user. While previous research has shown that data-driven methods can be used to reconstruct the correct postures, they usually require a large posture database, which greatly limit the usability for systems with constrained hardware such as game console. To solve this problem, we present a new probabilistic framework to enhance the accuracy of the postures live captured by Kinect. We adopt the Gaussian Process model as a prior to leverage position data obtained from Kinect and markerbased motion capture system. We also incorporate a temporal consistency term into the optimization framework to constrain the velocity variations between successive frames. To ensure that the reconstructed posture resembles the observed input data from Kinect when its tracking result is good, we embed joint reliability into the optimization framework. Experimental results demonstrate that our system can generate high quality postures even under severe self-occlusion situations, which is beneficial for real-time posture based applications such as motion-based gaming and sport training.

Filtered pose graph for efficient kinect pose reconstruction

“…Such a reliability estimation can then be integrated into a lazy learning framework to reconstruct the true pose [37]. Since the learning framework requires a large amount of poses, Gaussian Process is proposed to produce an abstract representation of the pose space and reduce the required database size [42]. However the unstructured nature of the database used during optimization cannot guarantee continuity of the resulting reconstructed poses and leads to consider a large number of candidate, some of them being inappropriate.…”

Kinect Posture Reconstruction Based on a Local Mixture of Gaussian Process Models

“…Compared with our previous work [9], we have significantly improved the spatial prediction algorithm. Firstly, with the newly proposed method based on local mixture of GP models, our method generates postures of similar quality to that of [9] with significantly less training data.…”

“…Compared with our previous work [9], we have significantly improved the spatial prediction algorithm. Firstly, with the newly proposed method based on local mixture of GP models, our method generates postures of similar quality to that of [9] with significantly less training data. Secondly, we design a new algorithm to incrementally update a specific local Gaussian Process in real time, which enables the system to adapt to run-time postures that are different from those in the database.…”

“…Lastly, because of the use of local models, our new framework only needs to consider a few local models that are close to an input posture rather than the whole database. Such an enhancement in efficiency allows us to combine all types of motion as a single database, while in [9] a separate database is built for each type of motion.…”

Usability of Corrected Kinect Measurement for Ergonomic Evaluation in Constrained Environment

“…To overcome this limitation, recent works have proposed to take the reliability of the Kinect data into account in the correction process. Reliability can then be integrated into a lazy learning framework (Shum et al, 2013) or a Gaussian model (Zhou et al, 2014;Liu et al, 2016) to reconstruct a more reliable pose.…”