Three-Dimensional Face Reconstruction Using Multi-View-Based Bilinear Model

Abstract

Face reconstruction is a popular topic in 3D vision system. However, traditional methods often depend on monocular cues, which contain few feature pixels and only use their location information while ignoring a lot of textural information. Furthermore, they are affected by the accuracy of the feature extraction method and occlusion. Here, we propose a novel facial reconstruction framework that accurately extracts the 3D shapes and poses of faces from images captured at multi-views. It extends the traditional method using the monocular bilinear model to the multi-view-based bilinear model by incorporating the feature prior constraint and the texture constraint, which are learned from multi-view images. The feature prior constraint is used as a shape prior to allowing us to estimate accurate 3D facial contours. Furthermore, the texture constraint extracts a high-precision 3D facial shape where traditional methods fail because of their limited number of feature points or the mostly texture-less and texture-repetitive nature of the input images. Meanwhile, it fully explores the implied 3D information of the multi-view images, which also enhances the robustness of the results. Additionally, the proposed method uses only two or more uncalibrated images with an arbitrary baseline, estimating calibration and shape simultaneously. A comparison with the state-of-the-art monocular bilinear model-based method shows that the proposed method has a significantly higher level of accuracy.

Application of three-dimensional reconstruction technology in dentistry: a narrative review

“…Its advantages in terms of application scenario limitations and imaging results are apparent, but due to its late development, the technology has not yet been adapted to the requirements of dentistry. In recent years, the research of this technology in terms of device hardware has gradually become a hot topic [ 212 ], but it is still in its initial stage.…”

Laplacian face blending

“…Fyffe et al 12 present a facial capture technique that combines the benefits of single‐shot and multi‐shot techniques by introducing a slight delay between the cameras in order to capture several lighting conditions while considering the subject static. Tian et al 13 present a method for accurately reconstructing faces using multiple viewpoints and shape priors. Fyffe et al 14 describe a video based facial capture which deforms a common template model to match multi‐view input images of the subject.…”

Making the Most of Single Sensor Information: A Novel Fusion Approach for 3D Face Recognition Using Region Covariance Descriptors and Gaussian Mixture Models

“…Recently, an approach that uses several 2D sensors to capture images at multiple viewpoints was proposed in [ 44 ]. This approach incorporates the feature prior constraint and the texture constraint to explore the implied 3D information of uncalibrated multiview images of a person’s face.…”