Detecting end-effectors on 2.5D data using geometric deformable models: Application to human pose estimation

Abstract

End-effectors are usually related to the location of limbs, and their reliable detection enables robust body tracking as well as accurate pose estimation. Recent innovation in depth cameras has re-stated the pose estimation problem. We focus on the information provided by these sensors, for which we borrow the name 2.5D data from the Graphics community. In this paper we propose a human pose estimation algorithm based on topological propagation. Geometric Deformable Models are used to carry out such propagation, implemented according to the Narrow Band Level Set approach. A variant of the latter method is proposed, including a density restriction which helps preserving the topological properties of the object under analysis. Principal end-effectors are extracted from a directed graph weighted with geodesic distances, also providing a skeletal-like structure describing human pose. An evaluation against reference methods is performed with promising results. The proposed solution allows a frame-wise end-effector detection, with no temporal tracking involved, which may be generalized to the tracking of other objects beyond human body.

Time Consistent Estimation of End-Effectors from RGB-D Data

“…Therefore, the proposed approach does not require any initialization or prior knowledge about the structure of the entity, which provides the ability to recover from estimation errors and the generality to estimate end-effectors for different entities. In our experiments, we proved that the proposed approach provides robustness for end-effector estimation, with its results outperforming the method in [13] in recall and average false positive rate while achieving similar results in terms of average distance error.…”

“…Each frame of the stream for both depth and color image is 640 × 480 pixels. In the rest of this section, we present the quantitative and qualitative results of the proposed approach while comparing it with the Restricted Narrow Band Level Set (R-NBLS) approach proposed in [13].…”

“…Following the work in [13], the external forces in the proposed system come from the foreground point cloud data C f g = {x i } ⊆ R 3 obtained in the previous step. The internal forces are defined as the expansion power.…”

“…Estimating endeffectors for 3D point cloud data requires representing the topology of the point cloud with a well organized structure. Thus, we exploit the approach proposed in [13] to describe the topology of point clouds. Then, a new strategy proposed in [2] for searching the end-effectors from its topology representation is integrated into the proposed system, providing a better performance than the strategy used in [13].…”

“…Suau el al. [13] employ a level set method in RGB-D data to exploit connectivities over the depth surface in order to extract a topological representation of point clouds. But these two approaches are not effective in solving the problem when deformable entities change the topology of the graph, in which case the detected extrema may not correspond to the position of the end effectors.…”