Facial Attribute Recognition With Feature Decoupling and Graph Convolutional Networks

Abstract

The objective of facial attribute recognition is to predict a set of labels for each face image. Similar to general multi-label image classification task, facial attribute recognition suffers from several typical problems: discriminative feature learning, handling the correlations among attributes, and imbalanced training data. In this paper, we propose a unified facial attribute recognition solution via feature decoupling and graph convolutional networks (GCN): 1) We utilize the orthonormal regularizer to constrain that each dimension of the general facial representation represents a certain visual pattern. Meanwhile, a learnable matrix is unveiled, which can convert general facial feature into attribute-specific representation through modeling the importance of different visual patterns. 2) To handle the correlations between facial attributes, we build a GCN to capture the label dependencies and map the nodes in the proposed GCN to a set of interdependent attribute classifiers. Besides, we normalize the weights in all classifiers to alleviate the influence of data imbalance. We have conducted extensive experiments on two benchmarks, and both the qualitative and quantitative evaluation results have demonstrated the effectiveness of the proposed method.

Synch-Graph: Multisensory Emotion Recognition Through Neural Synchrony via Graph Convolutional Networks

“…Applications of GCN are starting to emerge in computer vision in general and affective computing recently. Nian et al propose the use of GCN in facial features recognition (Nian et al 2019). They have used GCN for defining facial attributes such as hair colour, eyes or brow shape.…”

Facial attribute classification by deep mining inter‐attribute correlations

“…Compared to KT-MTL [12] and DMM-CNN [13], our approach exploits the correlations among attribute labels more deeply and reduces human intervention. Compared to Nian et al [37] and DPN [18], our method mitigates the effect of imbalanced data. It is important to note that only accuracy is -363 used when comparing the proposed solution with state-of-theart methods.…”

“…Mean accuracy (MA) over all attributes is a commonly used metric for the evaluation of classification. To appropriately evaluate the quality of different methods, we add three more instance-based evaluation metrics like Nian et al [37], that is, precision (Prec), recall (Rec), and F1-score (F1). Formally, these metrics can be calculated as: -359…”

Inducing Optimal Attribute Representations for Conditional GANs

“…Graph Convolutional Network (GCN) [24] is being popular in several tasks including link prediction [12], clustering [38], node classification [45]. Recent works on image classification [8] and face attributes classification [34] propose to use GCN to induce more discriminative representations of attributes by sharing information between the co-occurring attributes. Unlike these works, we propose to apply GCN to induce such higher-order representations of target categories for the generative neural networks and optimise it via end-to-end adversarial learning.…”