Recovering 3D human pose from monocular images

Abstract

Abstract-We describe a learning based method for recovering 3D human body pose from single images and monocular image sequences. Our approach requires neither an explicit body model nor prior labelling of body parts in the image. Instead, it recovers pose by direct nonlinear regression against shape descriptor vectors extracted automatically from image silhouettes. For robustness against local silhouette segmentation errors, silhouette shape is encoded by histogram-of-shape-contexts descriptors. We evaluate several different regression methods: ridge regression, Relevance Vector Machine (RVM) regression and Support Vector Machine (SVM) regression over both linear and kernel bases. The RVMs provide much sparser regressors without compromising performance, and kernel bases give a small but worthwhile improvement in performance. Loss of depth and limb labelling information often makes the recovery of 3D pose from single silhouettes ambiguous. We propose two solutions to this: the first embeds the method in a tracking framework, using dynamics from the previous state estimate to disambiguate the pose; the second uses a mixture of regressors framework to return multiple solutions for each silhouette. We show that the resulting system tracks long sequences stably, and is also capable of accurately reconstructing 3D human pose from single images, giving multiple possible solutions in ambiguous cases. For realism and good generalization over a wide range of viewpoints, we train the regressors on images resynthesized from real human motion capture data. The method is demonstrated on a 54-parameter full body pose model, both quantitatively on independent but similar test data, and qualitatively on real image sequences. Mean angular errors of 4-5 degrees are obtained -a factor of 3 better than the current state of the art for the much simpler upper body problem.

Holistic Human Pose Estimation with Regression Forests

“…The disadvantage of the exemplar-based approaches is the necessity for accurate matching of the whole body. To solve this problem, classification [9], regression [12] and segmentation-based [26] methods have been proposed. However, these methods can be sensitive to noisy input and cannot generalise to unknown poses.…”

“…On the other hand, the holistic approaches predict directly the body skeleton by learning a mapping between image features and skeletons [9][10][11][12]. These approaches usually face problems with occlusion or noise because they require complete data.…”

Monitoring Activities of Daily Living (ADLs) of Elderly Based on 3D Key Human Postures

“…The learning-based approaches avoid the need of an explicit 3D human body model. In [5], the authors propose a learning-based method for recovering 3D human body posture from single images and monocular image sequences. These 3D approaches are partially independent from the camera view point but they need to define many parameters to model the human posture.…”

Articulated tracking with manifold regularized particle filter

“…This approach is usually composed of two parts where feature extraction is followed by prediction using multivariate regression model. To obtain informative features simple techniques like binary silhouettes [1,14] as well as more sophisticated descriptors like histogram of oriented gradients or a HMAX model [3] were adapted. As a regression model a whole spectrum of different techniques were used, e.g., ridge regression and support vector machines [1], mixture of experts [10], gaussian processes [3], kernel information embedding [14].…”