Photo-Realistic Facial Details Synthesis From Single Image

Chen, Anpei; Chen, Zhang; Zhang, Guli; Mitchell, Kim; Yu, Jingyi

doi:10.1109/iccv.2019.00952

Cited by 116 publications

(94 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The other two results were generated using pre-trained deeplearning models. While the method from [49] estimates the shape of a face by inferring the shape parameters of a 3DMM, a detailed facial shape with a UV displacement map is generated in [36]. Although our method is based on iterative linear optimization, the quality of the recovered 3D shapes is comparable to that of the latest methods based on deeplearning.…”

Section: B 3d Shape Recoverymentioning

confidence: 98%

“…Recently, a UV map, which is mapping of 3D mesh to the 2D image, was used for inferring 3D displacement from 2D image [34], [35]. The latest prominent results by using a UV map combined with a CNN was proposed in [36].…”

Section: Related Workmentioning

confidence: 99%

“…Comparison of results for 3D facial shape recovery from the Bosphorus database. We compare our proposed method to the methods proposed by Jiang et al [17], Deng et al [49], and Chen et al [36]. Note that ground truth was constructed by meshing the noisy depth map provided in the database.…”

Section: Rs-mse S-ssimmentioning

confidence: 99%

See 2 more Smart Citations

Local Spherical Harmonics for Facial Shape and Albedo Estimation

et al. 2020

View full text Add to dashboard Cite

In this paper, we present a novel facial albedo and 3D shape recovery method with a local spherical harmonic illumination model. From a face in an image, the proposed method can produce a highquality 3D shape and albedo using a novel parameterization of local illuminations. Because a facial shape is partially convex, a single spherical harmonics is generally used for the illumination of a face within a constrained illumination environment. However, when a facial image is captured in an unconstrained scene, the illumination is inappropriately estimated due to the presence of shadow and specular reflections. To address this issue, we propose a novel local spherical harmonic illumination model for representing the illumination of a face. Unlike the existing parameterization of local illumination, our local spherical harmonic illumination model utilizes a smooth weight function for the seamless representation of natural illumination. Therefore, the albedo and shape information in an image can be precisely estimated using the first-order spherical harmonics. For accurate estimation of albedo, we also utilize facial albedo statistics to prevent the estimated albedo from becoming biased toward input image. Furthermore, we developed an accurate and reliable 3D shape reconstruction method from a normal map based on tetrahedron-based deformation. Comparing to the Laplacian deformation based method, our method is applicable to any mesh regardless of its structure. Through rigorous experiments, we demonstrate that the proposed local spherical harmonic illumination model is effective in estimating the complex illumination and can recover a highquality facial albedo and 3D shape.

show abstract

Section: B 3d Shape Recoverymentioning

confidence: 98%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Local Spherical Harmonics for Facial Shape and Albedo Estimation

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Although there have been some studies involving video processing, such as video generation [115], video colorization [116], [117], video inpainting [118], motion transfer [119], and facial animation synthesis [120]- [123], the research on video using GANs is limited. In addition, although GANs have been applied to the generation and synthesis of 3D models, such as 3D colorization [124], 3D face reconstruction [125], [126], 3D character animation [127], and 3D textured object generation [128], the results are far from perfect. At present, GANs are still based on large amounts of training data.…”

Section: B Future Opportunitiesmentioning

confidence: 99%

A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks

et al. 2020

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have achieved impressive results in various image synthesis tasks, and are becoming a hot topic in computer vision research because of the impressive performance they achieved in various applications. In this paper, we introduce the recent research on GANs in the field of image processing, including image synthesis, image generation, image semantic editing, image-to-image translation, image super-resolution, image inpainting, and cartoon generation. We analyze and summarize the methods used in these applications which have improved the generated results. Then, we discuss the challenges faced by GANs and introduce some methods to deal with these problems. We also preview some likely future research directions in the field of GANs, such as video generation, facial animation synthesis and 3D face reconstruction. The purpose of this review is to provide insights into the research on GANs and to present the various applications based on GANs in different scenarios. INDEX TERMS Generative adversarial networks, image synthesis, image-to-image translation, image editing, cartoon generation.

show abstract

“…The origin of visual generation dates back to [40], where multiple networks are jointly trained in an adversarial manner. Subsequent works generate images in specific domains such as face [41][42][43], person [44][45][46], as well as generic domains [47,48]. From the perspective of inputs, the generation can also be treated as conditioning on different information, e.g.…”

Section: I I L a N G U A G E T O V I S I O Nmentioning

confidence: 99%

Vision and language: from visual perception to content creation

Mei

Zhang

Yao

2020

SIP

View full text Add to dashboard Cite

Vision and language are two fundamental capabilities of human intelligence. Humans routinely perform tasks through the interactions between vision and language, supporting the uniquely human capacity to talk about what they see or hallucinate a picture on a natural-language description. The valid question of how language interacts with vision motivates us researchers to expand the horizons of computer vision area. In particular, “vision to language” is probably one of the most popular topics in the past 5 years, with a significant growth in both volume of publications and extensive applications, e.g. captioning, visual question answering, visual dialog, language navigation, etc. Such tasks boost visual perception with more comprehensive understanding and diverse linguistic representations. Going beyond the progresses made in “vision to language,” language can also contribute to vision understanding and offer new possibilities of visual content creation, i.e. “language to vision.” The process performs as a prism through which to create visual content conditioning on the language inputs. This paper reviews the recent advances along these two dimensions: “vision to language” and “language to vision.” More concretely, the former mainly focuses on the development of image/video captioning, as well as typical encoder–decoder structures and benchmarks, while the latter summarizes the technologies of visual content creation. The real-world deployment or services of vision and language are elaborated as well.

show abstract

Photo-Realistic Facial Details Synthesis From Single Image

Cited by 116 publications

References 63 publications

Local Spherical Harmonics for Facial Shape and Albedo Estimation

Local Spherical Harmonics for Facial Shape and Albedo Estimation

A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks

Vision and language: from visual perception to content creation

Contact Info

Product

Resources

About