Human Shape and Pose Tracking Using Keyframes

Abstract

This paper considers human tracking in multi-view setups and investigates a robust strategy that learns online key poses to drive a shape tracking method. The interest arises in realistic dynamic scenes where occlusions or segmentation errors occur. The corrupted observations present missing data and outliers that deteriorate tracking results. We propose to use key poses of the tracked person as multiple reference models. In contrast to many existing approaches that rely on a single reference model, multiple templates represent a larger variability of human poses. They provide therefore better initial hypotheses when tracking with noisy data. Our approach identifies these reference models online as distinctive keyframes during tracking. The most suitable one is then chosen as the reference at each frame. In addition, taking advantage of the proximity between successive frames, an efficient outlier handling technique is proposed to prevent from associating the model to irrelevant outliers. The two strategies are successfully experimented with a surface deformation framework that recovers both the pose and the shape. Evaluations on existing datasets also demonstrate their benefits with respect to the state of the art.

Toward user-specific tracking by detection of human shapes in multi-cameras

“…Data is explained by Gaussian Mixture Models (GMM) in an Expectation-Maximization (EM) manner. Huang et al [15] follow a similar concept, but aggregate the outlier likelihood over every Gaussian component and offer better robustness. All these generative methods are highly likely to fail in large deformations.…”

“…Among the vast literature on human pose estimation [19], we focus on top-down approaches that assume a 3D model and deform it according to input data, either directly with pixels as in [12,18,25], or with 3D points as in [6,8,15]. These methods typically decompose into two main steps: (i) data association, where observations are associated to the model, and (ii) deformation estimation, where deformation parameters are estimated given the associations.…”

“…[6,8], reducing error accumulations and hence increasing the stability. The contribution of this work is two-fold: (1) a one-shot correspondence inference with complete 3D input (rather than 2.5D as in [22,27]), which leads to (2) a hybrid subjectspecific human pose and shape capture method that relies little on former results (unlike other ICP-like methods [6,12,15,18]), holding the ability to recover from drifting.…”

“…Most of these model-based methods [6,12,15,18,29] can be viewed as extensions of Iterative-Closest-Point (ICP) framework [5], which attempts to explain newly observed data using the previous outcomes. As long as the initialization is close to the optimum solution, it is able to produce .…”

“…In particular, it has recently demonstrated a good success in marker-less human motion capture (mocap). Numerous approaches assume a user-specific reference surface, and the objective is to recover the skeletal poses [29], surface shapes [6], or both simultaneously [15].…”