An Augmented Reality Virtual Glasses Try-On System

Azevedo, Pedro Miguel Henriques; Santos, Thiago Oliveira dos; Aguiar, Edilson de

doi:10.1109/svr.2016.12

Cited by 8 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Virtual make-up can be applied based on a reflectance/shading decomposition [Li et al 2014b[Li et al , 2015b. Similar techniques enable the try-on of accessories, e.g., eyeglasses [Azevedo et al 2016;Niswar et al 2011].…”

Section: Virtualmentioning

confidence: 99%

3D Morphable Face Models—Past, Present, and Future

et al. 2020

View full text Add to dashboard Cite

In this article, we provide a detailed survey of 3D Morphable Face Models over the 20 years since they were first proposed. The challenges in building and applying these models, namely, capture, modeling, image formation, and image analysis, are still active research topics, and we review the state-of-the-art in each of these areas. We also look ahead, identifying unsolved challenges, proposing directions for future research, and highlighting the broad range of current and future applications.

show abstract

Section: Virtualmentioning

confidence: 99%

3D Morphable Face Models—Past, Present, and Future

et al. 2020

View full text Add to dashboard Cite

show abstract

“…In general, AR renderers are composed of one or several scene perception modules whose role is to estimate relevant scene information from the source image, that is then passed to a rendering module. For instance, many portraits based AR applications are based on facial landmarks estimation [KS14] used to compute the position of a synthetic object such as glasses that are then blended on the face image [ADSDA16]. Similarly, other popular scene perception methods for augmented reality focus on hand tracking [WMB ∗ 20, ZBV ∗ 20], body pose estimation [BGR ∗ 20], hair segmentation [LCP ∗ 18], or scene depth estimation [KRH21].…”

Section: Related Workmentioning

confidence: 99%

Real‐time Virtual‐Try‐On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Renderers

Kips

Jiang²,

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

Augmented reality applications have rapidly spread across online retail platforms and social media, allowing consumers to virtually try-on a large variety of products, such as makeup, hair dying, or shoes. However, parametrizing a renderer to synthesize realistic images of a given product remains a challenging task that requires expert knowledge. While recent work has introduced neural rendering methods for virtual try-on from example images, current approaches are based on large generative models that cannot be used in real-time on mobile devices. This calls for a hybrid method that combines the advantages of computer graphics and neural rendering approaches. In this paper, we propose a novel framework based on deep learning to build a real-time inverse graphics encoder that learns to map a single example image into the parameter space of a given augmented reality rendering engine. Our method leverages self-supervised learning and does not require labeled training data, which makes it extendable to many virtual try-on applications. Furthermore, most augmented reality renderers are not differentiable in practice due to algorithmic choices or implementation constraints to reach real-time on portable devices. To relax the need for a graphics-based differentiable renderer in inverse graphics problems, we introduce a trainable imitator module. Our imitator is a generative network that learns to accurately reproduce the behavior of a given non-differentiable renderer. We propose a novel rendering sensitivity loss to train the imitator, which ensures that the network learns an accurate and continuous representation for each rendering parameter. Automatically learning a differentiable renderer, as proposed here, could be beneficial for various inverse graphics tasks. Our framework enables novel applications where consumers can virtually try-on a novel unknown product from an inspirational reference image on social media. It can also be used by computer graphics artists to automatically create realistic rendering from a reference product image. CCS Concepts• Computing methodologies → Computer vision; Machine learning; Computer graphics; Recently, a novel family of methods based on neural rendering has introduced the task of image based virtual try-on, which brings new perspectives for this problem [TFT * 20]. This task consists in extracting a product appearance from a single reference image and synthesizing it on the image of another person. However, existing methods in this domain are often based on large genera-

show abstract

“…21). In fact, many AR techniques can be adopted for face data augmentation, such as the virtual mirror [129] proposed for facial geometric alteration, the magic mirror [130] used for makeup or accessories try-on, the Beauty e-Experts system [131] designed for hairstyle and facial makeup recommendation and synthesis, the virtual glasses try-on system [132], etc. More experiences of data augmentation by AR can be found in [133], [134], and [135].…”

Section: E Augmented Realitymentioning

confidence: 99%

A Survey on Face Data Augmentation

Wang,

Lian

2019

Preprint

View full text Add to dashboard Cite

The quality and size of training set have great impact on the results of deep learning-based face related tasks. However, collecting and labeling adequate samples with high quality and balanced distributions still remains a laborious and expensive work, and various data augmentation techniques have thus been widely used to enrich the training dataset. In this paper, we systematically review the existing works of face data augmentation from the perspectives of the transformation types and methods, with the state-of-the-art approaches involved. Among all these approaches, we put the emphasis on the deep learning-based works, especially the generative adversarial networks which have been recognized as more powerful and effective tools in recent years. We present their principles, discuss the results and show their applications as well as limitations. Different evaluation metrics for evaluating these approaches are also introduced. We point out the challenges and opportunities in the field of face data augmentation, and provide brief yet insightful discussions.

show abstract

An Augmented Reality Virtual Glasses Try-On System

Cited by 8 publications

References 19 publications

3D Morphable Face Models—Past, Present, and Future

3D Morphable Face Models—Past, Present, and Future

Real‐time Virtual‐Try‐On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Renderers

A Survey on Face Data Augmentation

Contact Info

Product

Resources

About