Examples-Rules Guided Deep Neural Network for Makeup Recommendation

Abstract

In this paper, we consider a fully automatic makeup recommendation system and propose a novel examples-rules guided deep neural network approach. The framework consists of three stages. First, makeup-related facial traits are classified into structured coding. Second, these facial traits are fed in- to examples-rules guided deep neural recommendation model which makes use of the pairwise of Before-After images and the makeup artist knowledge jointly. Finally, to visualize the recommended makeup style, an automatic makeup synthesis system is developed as well. To this end, a new Before-After facial makeup database is collected and labeled manually, and the knowledge of makeup artist is modeled by knowledge base system. The performance of this framework is evaluated through extensive experimental analyses. The experiments validate the automatic facial traits classification, the recommendation effectiveness in statistical and perceptual ways and the makeup synthesis accuracy which outperforms the state of the art methods by large margin. It is also worthy to note that the proposed framework is a pioneering fully automatic makeup recommendation systems to our best knowledge.

Virtual Makeup Based on Golden Sample Search

“…Here, we compare our proposed method with the methods presented in [1][2][3][4][5][6][7] in terms of the attribute of the techniques used. We analyze the existing methods and summarize them into three categories according to makeup characteristics listed in Table 1.…”

“…Because the makeup synthesis works on the whole face, makeup part selection is not available and color tuning is limited by the example face. Our proposed method and [6][7] are Category III and have the most degrees of freedom in all aspects as they provide the ability of individual facial-part processing. As a result, the methods of Category III may yield better makeup effects in color, texture and shape.…”

“…As a result, the methods of Category III may yield better makeup effects in color, texture and shape. We make comparison for the methods in [1][2][3][4][5][6][7] and our method in terms of four techniques used, as listed in Table 2. In facial part location, [1-2] use skin color segmentation and all of the rest use landmark detection to improve performance.…”

“…In [7], a deep neural network is proposed to build a recommendation model that is trained from examples and knowledge base rules jointly. Each makeup-related facial trait is classified into certain classes and coded as a feature vector.…”

Anti-Makeup: Learning A Bi-Level Adversarial Network for Makeup-Invariant Face Verification

“…Makeup related studies, such as makeup recommendation (Alashkar et al 2017), have become more popular than ever. However, relatively less articles pay attention on the challenge of makeup impact on face verification.…”