Personalized Face Modeling for Photorealistic Synthesis

Abstract

IntroductionFaces are one of the most important objects in our daily life. People heavily depend on a face to recognize the identity, gender, age, emotion of person and also perceive the attractiveness and other subjective properties of individual. Such a unique role of faces is associated with a wide range of applications such as forensics, game, movie special effects, virtual and augmented reality and many others. Accordingly, face modeling has received a great attention from research communities for decades and a huge amount of literature suggest the solution to this problem.Existing work aims to either extract the intrinsic attributes of the face (e.g. identity, age, emotion, shape, skin reflectance) or to synthesize the realistic face. The first group develops the powerful parameterized face model and applies this model into the input to derive the personalized model parameter. The input can vary upon the application scenario and it is normally as few as a single face image with some extra information (e.g. a depth image, a collection of the same person's face, etc.) if available. The major challenge of this problem is to accomplish the robust estimation of model parameters. It is because the limited number of inputs leads the inference ill-posed unless incorporating the strong constrains. To implant the strong regularization power into the model, it is important to build a concise appearance model with a reduced number of parameters. In this way, the inference problem becomes feasible.The other group focuses on constructing a highly realistic face model for the synthesis purpose. To achieve the realism in face model, they often increase a number of parameters for generating the complex illumination with details on skin (e.g. subsurface scattering and freckles). Especially, the subsurface scattering effects are the most challenging aspect of human skin. As a result, it is often required to capture up to few thousands of input images for constructing the personalized face model. This process requires not only many input images but also a specialized equipment under the studio-like environment because the scattering component is rarely factorized from regular images.Therefore, the framework of realistic face modeling is often developed with its capturing system. Recent approaches in face modeling are further developed to achieve the both robustness and realism. For example, Shim (33) suggested the face specular map to accomplish the robust parameter estimation and also shows the improved performance in realistic facial synthesis. More recently, unstructured face images are utilized in 3D face Faces play a key role in revealing the personalized attributes such as the identity, emotion, health condition, etc. Due to the importance of faces, computer-assisted face modeling and reconstruction have been actively studied both in computer vision and graphics community. Especially, face reconstruction and realistic face synthesis are well-grounded research problems and various approaches have been proposed during the...