Piecewise Bézier Space: Recovering 3D Dynamic Motion From Video

Abstract

In this paper we address the problem of jointly retrieving a 3D dynamic shape, camera motion, and deformation grouping from partial 2D point trajectories in a monocular video. To this end, we introduce a union of piecewise Bézier subspaces with enforcing continuities to model 3D motion. We show that formulating the problem in terms of piecewise curves, allows for a better physical interpretation of the resulting priors and a more accurate representation of the motion. An energybased formulation is presented to solve the problem in an unsupervised, unified, accurate and efficient manner, by means of the use of augmented Lagrange multipliers. We thoroughly validate the approach on a wide variety of human video sequences, including those cases with noisy and missing observations, and providing more accurate joint estimations than state-of-the-art approaches.

Spline Human Motion Recovery

“…To solve that, splines can be considered to obtain a piecewise smooth curve that is coded by low-order -up to a degree of 3-polynomials. In this context, maybe the most standard way is the use of low-order Bézier curves [15]. For Bézier curves, the solution is contained in the convex hull of the control polygon and, the use of quadratic or cubic functions is a direct consequence of the number of control points to encode the global curve.…”

“…tions is employed for every modality instead of a combination of quadratic and cubic pieces that are used for piecewise Bézier curves [15]. In general terms, the number of pieces is bigger and, therefore, the global curve is more local-aware controllable, while C 1 and C 2 continuities between consecutive pieces can be automatically enforced without including any additional constraint.…”

“…One possibility to handle the previous problem is the use of trajectory-based models [6,14,15,26,27], where the location of each point coordinate over time is encoded by a linear combination of K low-frequency basis vectors. These methods were first applied in a global manner without being useful to recover piece-varying motions and later, in a local fashion by means of the use of piecewise Bézier curves [15]. Despite providing striking results and local controllability, the global curve could not be smooth enough as some continuities need to be enforced by additional constraints.…”

“…To sort out this limitation, we propose the use of B-splines or Catmull-Rom functions, achieving so a more compact representation where the continuities are automatically enforced. To this end, and following [15], we consider N 3K-dimensional vectors as κ n = [p n x1 , . .…”

“…where the problem is handled in a global manner. More recently, these methods were extended to piece-varying motion, where the trajectory basis is local-aware, more controllable but it needs to impose restrictive continuity constraints to smoothly connect the pieces [15]. The low-rank constraint has also been imposed by directly minimizing the rank of a matrix representing the 4D shape, considering the data lie in a single [16,17], in a union of temporal [18,19], in a dual union of spatio-temporal [20,21], or in multiple [22] subspaces.…”