What Shape Are Dolphins? Building 3D Morphable Models from 2D Images

Abstract

3D morphable models are low-dimensional parameterizations of 3D object classes which provide a powerful means of associating 3D geometry to 2D images. However, morphable models are currently generated from 3D scans, so for general object classes such as animals they are economically and practically infeasible. We show that, given a small amount of user interaction (little more than that required to build a conventional morphable model), there is enough information in a collection of 2D pictures of certain object classes to generate a full 3D morphable model, even in the absence of surface texture. The key restriction is that the object class should not be strongly articulated, and that a very rough rigid model should be provided as an initial estimate of the “mean shape.” The model representation is a linear combination of subdivision surfaces, which we fit to image silhouettes and any identifiable key points using a novel combined continuous-discrete optimization strategy. Results are demonstrated on several natural object classes, and show that models of rather high quality can be obtained from this limited information.

3D scanning deformable objects with a single RGBD sensor

“…Note that some special care has to be taken to allow the Levenberg-Marquardt algorithm to interact with a surface coordinate variable u ∈ U [16,2]. Such a variable has the atypical parameterization u = (τ, u, v) where τ is discrete (a triangle ID), and (u, v) are real valued coordinates in the unit triangle.…”

Learning an efficient model of hand shape variation from depth images

“…Also, their optimization steps are sequential (block coordinate descent) rather than simultaneous, which may result in a poor local optimum being obtained. Cashman and Fitzgibbon [9] demonstrate that morphable models using subdivision surfaces can be learned from extremely limited data (30 silhouette images). However their approach does not separate shape and pose, and neither learns a parametric shape basis.…”

“…2 penalizing deviations by the large weight λ skin in (9). In order to ensure that the skinning weights remain non-negative, we simply parameterize the weight of vertex m with bone b in the log domain asα bm = log(α bm ).…”

Guiding Image Manipulations using Shape‐appearance Subspaces from Co‐alignment of Image Collections

“…Images from a certain class, such as faces, do not cover the entire space but a lower-dimensional manifold which is likely related to the human mental representation of this class [SL00]. This idea was first proposed for human faces, both in 2D [TP91, CET01] and 3D [BV99], 3D human bodies [ACP03] or other specialized 3D shapes [CF13], where the manifold is approximated using principal component analysis (PCA).…”

“…Their applications range from faces [BV99] over human body poses [ACP03] to other objects such as demonstrated for sea animals by Cashman and Fitzgibbon [CF13]. Recently, the idea of finding subspaces has been extended to 3D objects when either given a collection of 3D objects [OLGM11] or even just a collection of images [PFZG10].…”