Using a Deformation Field Model for Localizing Faces and Facial Points under Weak Supervision

Abstract

Face detection and facial points localization are interconnected tasks. Recently it has been shown that solving these two tasks jointly with a mixture of trees of parts (MTP) leads to state-of-the-art results. However, MTP, as most other methods for facial point localization proposed so far, requires a complete annotation of the training data at facial point level. This is used to predefine the structure of the trees and to place the parts correctly. In this work we extend the mixtures from trees to more general loopy graphs. In this way we can learn in a weakly supervised manner (using only the face location and orientation) a powerful deformable detector that implicitly aligns its parts to the detected face in the image. By attaching some reference points to the correct parts of our detector we can then localize the facial points. In terms of detection our method clearly outperforms the state-of-the-art, even if competing with methods that use facial point annotations during training. Additionally, without any facial point annotation at the level of individual training images, our method can localize facial points with an accuracy similar to fully supervised approaches.

Unsupervised Learning of Object Landmarks by Factorized Spatial Embeddings

“…There is an extensive literature on landmark detectors, particularly for faces. Examples include Active Appearance Models [11], along with subsequent improvements [44,12] and others using templates [51] or parts [80]. Other approaches directly regress the landmark coordinates [59,14,10,52].…”

Deep multi-center learning for face alignment

“…Asthana et al [16] used regression techniques to learn functions from response maps to shapes, in which the response map has stronger robustness and generalization ability than texture based features of AAM. Pedersoli et al [17] developed the mixture of trees of parts method by extending the mixtures from trees to graphs, and learned a deformable detector to align its parts to faces. However, these templates are not complete enough to cover complex variations, which are difficult to be generalized to unseen faces.…”

300 Faces In-The-Wild Challenge: database and results

“…Various methods have been proposed in the literature for the task of landmark localization under semi-supervised or weakly-supervised settings [33,34,35,36]. However, there are two major limitations of these methods.…”

“…The purpose of these masks is to indicate which pixels belong to the facial area and the only way to produce them is by manually annotating each image. In [34], the training procedure requires as input the orientation of each face depicted in the training images. Secondly, and most importantly, existing methods, such as [35] and [36], have only been applied on images that are captured under controlled conditions.…”