A Survey of Deep Facial Attribute Analysis

Abstract

Facial attribute analysis has received considerable attention when deep learning techniques made remarkable breakthroughs in this field over the past few years. Deep learning based facial attribute analysis consists of two basic sub-issues: facial attribute estimation (FAE), which recognizes whether facial attributes are present in given images, and facial attribute manipulation (FAM), which synthesizes or removes desired facial attributes. In this paper, we provide a comprehensive survey of deep facial attribute analysis from the perspectives of both estimation and manipulation.First, we summarize a general pipeline that deep facial attribute analysis follows, which comprises two stages: data preprocessing and model construction. Additionally, we introduce the underlying theories of this twostage pipeline for both FAE and FAM. Second, the datasets and performance metrics commonly used in facial attribute analysis are presented. Third, we create a taxonomy of state-of-the-art methods and review deep FAE and FAM algorithms in detail. Furthermore, several additional facial attribute related issues are introduced, as well as relevant real-world applications. Finally, we discuss possible challenges and promising future research directions.

Estimation of Semantic Impressions from Portraits

“…In recent years, it has become possible for users to manipulate portraits relatively easily by using web service smartphone applications such as BeautyPlus † , SNOW † † , and VSCO † † † , even if they have little experience in editing portraits. Meanwhile, researchers have developed a system that can automatically estimate a person's attributes and generate new portraits based on those attributes [31].…”

Optimizing Few-Shot Learning based on Variational Autoencoders

“…In the last few years, face attribute analysis has made considerable progress along with deep learning. Facial attribute analysis based on deep learning includes two research directions: (1) facial attribute estimation (FAE), which is used to identify whether there are facial attributes in a given image, and (2) FAM, which is used to synthesize or remove specific facial attributes [70]. In this research, we focus on FAM.…”

“…The latest progress of the FAM method is primarily built around the deep learning generative models, of which GANs and VAEs are the two most popular models. There are two main methods to get the FAM on generative models: (1) model-based methods and (2) extra-condition-based methods [70]. Furthermore, there are two kinds of extra-condition-based methods: (1) attribute vectors as extra conditions, which, with extra input vector, rely on simple linear interpolation, and ( 2) reference exemplars as extra conditions, which directly learn the image-to-image translation along with attributes that is popular for unsupervised disentanglement learning.…”

Augmented binary multi-labeled CNN for practical facial attribute classification

“…This contributes and plays an important role in security [1], [2] like access control for PCs or smartphone, video surveillance, criminal authentication, face sketch [3] and face photo for the law enforcement application [4]. The main property of FAC [5] is to predict multiple face features, state and emotion [6] on the given image or face portrait. Various algorithms have reached an excellent result on multiple levels for FAC, either apply directly CNN [7] models to extract face features, or using methods for improving the learning by distributing the attributes into two categories: objective attributes like wearing a hat, eyeglasses, bangs and subjective ones like smiling, big lips [8].…”