A Novel Reconstruction and Tracking of 3D-Articulated Human Body from 2D Point Correspondences of a Monocular Image Sequence

Abstract

SUMMARYA novel method is proposed to estimate the 3D relative positions of an articulated body from point correspondences in an uncalibrated monocular image sequence. It is based on a camera perspective model. Unlike previous approaches, our proposed method does not require camera parameters or a manual specification of the 3D pose at the first frame, nor does it require the assumption that at least one predefined segment in every frame is parallel to the image plane. Our work assumes a simpler assumption, for example, the actor stands vertically parallel to the image plane and not all of his/her joints lie on a plane parallel to the image plane in the first frame. Input into our algorithm consists of a topological skeleton model and 2D position data on the joints of a human actor. By geometric constraint of body parts in the skeleton model, 3D relative coordinates of the model are obtained. This reconstruction from 2D to 3D is an ill-posed problem due to non-uniqueness of solutions. Therefore, we introduced a technique based on the concept of multiple hypothesis tracking (MHT) with a motion-smoothness function between consecutive frames to automatically find the optimal solution for this ill-posed problem. Since reconstruction configurations are obtained from our closed-form equation, our technique is very efficient. Very accurate results were attained for both synthesized and real-world image sequences. We also compared our technique with both scaled-orthographic and existing perspective approaches. Our proposed method outperformed other approaches, especially in scenes with strong perspective effects and difficult poses. key words: perspective model, human skeleton, 3D tracking, MHT

A dynamic portrait segmentation by merging colors and depth information

“…The key issue of these technology is image processing and the processing content always is on these portrait which is a complicate problem to distinguish the foreground image segment from the random environment. The body image segmentation still has a big challenge, although a lot of scholars have proposed many methods to solve it such as the significant object segmentation for the shallow depth-offield images [3], classifying the object clusters via particle swarm algorithm according to the images color features [4][5][6], the image segment based on the shape by the photometric stereo technique [7][8][9], or using the triangulation to find and segment the portrait [10][11][12][13] and so on. However the image features must be proceeded under the appointed conditions.…”