Tracking 3D Object using Flexible Models

Abstract

This article proposes a flexible tracker which can estimate motions and deformations of 3D objects in images by considering their appearances as nonrigid surfaces. In this approach, a flexible model is built by matching local features (key-points) over training sequences and by learning the deformations of a spline based model. A statistical model captures the variations of object appearance caused by 3D pose variations. Visual tracking is then possible, for each new frame, by matching the local features of the model with the image features, according to their local appearances and the constraints provided by the flexible model. This approach is demonstrated on real-world image sequences.

Generalized Thin-Plate SplineWarps

“…RP-Warps depend on the fundamental matrix F and a depth vector δ. ϑ(W RP , {δ, F}) is an NLS problem, for several reasons: (i) δ and F are coupled in the homogeneous expression (12) of the warp, (ii) finding the affine coordinates of the transferred point requires a division and (iii) the fundamental matrix must fulfill a nonlinear rankdeficiency constraint 5 . Similarly to the algorithm for the RA-Warps, we use a two-step initialization procedure.…”

“…[1]. Standard TPS warps have recently been used as simple parametric warps for images of rigid 3D surfaces by Wills and Belongie [8] and Masson et al [5], for respectively wide-baseline matching and object tracking.…”

Generalized Thin-Plate Spline Warps

“…The standard TPS warp has recently been used as a simple parametric warp for images of a rigid and smooth 3D surface in Wills and Belongie (2004) and Masson et al (2005), for respectively wide-baseline image matching and object tracking.…”

Efficient Model-based 3D Tracking by Using Direct Image Registration

“…We can augment this model by adding further texture priors either from the image stream [Cobzas et al, 2009;Muñoz et al, 2005;Sepp and Hirzinger, 2003;Vacchetti et al, 2004;Xiao et al, 2004a;Zimmermann et al, 2006], or from and external source (e.g. a 3D scanner or a texture mosaic) [Hong and Chung, 2007;La Cascia et al, 2000;Masson et al, 2004Masson et al, , 2005Pressigout and Marchand, 2007;Romdhani and Vetter, 2003].…”

“…The motion model is tightly coupled to the target model: it is usually represented by a geometric transformation that maps the coordinates of the target model into a different set of coordinates. For a planar target, these geometric transformations are typically affine [Hager and Belhumeur, 1998], homographic [Baker and Matthews, 2004;Buenaposada and Baumela, 1999], or spline-based warps [Bartoli and Zisserman, 2004;Brunet et al, 2009;Lester and Arridge, 1999;Masson et al, 2005]. For actual 3D targets, the geometric warps account for computing the rotation and translation of the object using a 6 degreeof-freedom (dof) rigid body transformation [Cipolla and Drummond, 1999;La Cascia et al, 2000;Marchand et al, 1999;Sepp and Hirzinger, 2003].…”