Separable Spatiotemporal Priors for Convex Reconstruction of Time-Varying 3D Point Clouds

Abstract

Reconstructing 3D motion data is highly under-constrained due to several common sources of data loss during measurement, such as projection, occlusion, or miscorrespondence. We present a statistical model of 3D motion data, based on the Kronecker structure of the spatiotemporal covariance of natural motion, as a prior on 3D motion. This prior is expressed as a matrix normal distribution, composed of separable and compact row and column covariances. We relate the marginals of the distribution to the shape, trajectory, and shape-trajectory models of prior art. When the marginal shape distribution is not available from training data, we show how placing a hierarchical prior over shapes results in a convex MAP solution in terms of the trace-norm. The matrix normal distribution, fit to a single sequence, outperforms state-of-the-art methods at reconstructing 3D motion data in the presence of significant data loss, while providing covariance estimates of the imputed points.

Discrete Laplace Operator Estimation for Dynamic 3D Reconstruction

“…Along these lines, Avidan and Shashua [6] estimate dynamic geometry from 2D observations of points constrained to linear and conical motions. However, under the assumption of dense temporal motion sampling, the concept of motion smoothness has been successfully exploited [25,26,45,46,35,42,43,36,30,31]. Park et al [25] triangulate 3D point trajectories by the linear combination of Direct Cosine Transform trajectory bases with the constraint of a reprojection system.…”

DUST: Dual Union of Spatio-Temporal Subspaces for Monocular Multiple Object 3D Reconstruction

“…On top of these shape models, additional spatial [24] or temporal [1,8,26] smoothness constraints have also been considered. Low-rank models have been extended to the temporal domain, by fitting point trajectories to a series of predefined basis [6,32,38], to shape-and-temporal composite domains [21,22,35], and to the space of forces that induce the deformations [3].…”

“…There are several alternatives and approximations for doing so, e.g., strategies that enforce smooth trajectories [21,22], methods based on trace-norm minimization that assume the rank of the subspace a priori [16,35] or techniques based on Procrustes analysis [24]. Of course, for easier scenarios, G could also be recovered using a few background rigid points and then applying rigid factorization [32].…”

Recovering Pose and 3D Deformable Shape from Multi-instance Image Ensembles

“…In [32], trajectory priors were used in terms of 3D point differentials. Subsequent works have combined shape and trajectory constraints [6,33,34]. More recently, both low-rank shape and trajectory subspaces have been linked to a force subspace, giving them a physical interpretation [9].…”