Towards Multi-view and Partially-Occluded Face Alignment

Abstract

We present a robust model to locate facial landmarks under different views and possibly severe occlusions. To build reliable relationships between face appearance and shape with large view variations, we propose to formulate face alignment as an 1 -induced Stagewise Relational Dictionary (SRD) learning problem. During each training stage, the SRD model learns a relational dictionary to capture consistent relationships between face appearance and shape, which are respectively modeled by the pose-indexed image features and the shape displacements for current estimated landmarks. During testing, the SRD model automatically selects a sparse set of the most related shape displacements for the testing face and uses them to refine its shape iteratively. To locate facial landmarks under occlusions, we further propose to learn an occlusion dictionary to model different kinds of partial face occlusions. By deploying the occlusion dictionary into the SRD model, the alignment performance for occluded faces can be further improved. Our algorithm is simple, effective, and easy to implement. Extensive experiments on two benchmark datasets and two newly built datasets have demonstrated its superior performances over the state-of-the-art methods, especially for faces with large view variations and/or occlusions.

Sign-Correlation Partition Based on Global Supervised Descent Method for Face Alignment

“…Their ERT achieves millisecond performance and can handle partial or uncertain labels, but the correlation of shape parameters is little taken into account. Instead of least squares regression, Xing et al [13] learn sparse Stagewise Relational Dictionary (SRD) between facial appearances and shapes, which improves the robustness under different views and severe occlusions. Some recent research aims at choosing or learning shape-indexed features.…”

“…Some methods among them have begun to deal with the impact of poses, like RPCR [12], SRD [13], CCR [31], hierarchical localization [34] and coarse-tofine searching [30]. However, few of them can give a clear interpretation for the correlation between poses and feature or shapes.…”

“…Some of them can handle partial occlusions [12][13][14]. Some works mainly aim at speeding up the prediction process while keeping high accuracy [13,15]. The choice and learning of shape-indexed features are also studied [15][16][17].…”

“…Though their scheme works well for face tracking and pose estimation, it is not suitable for face alignment across various poses, because the ground truth of shapes or features are unknown and it is not able to find approximation due to lack of previous frame. A few recent works [12,13,[28][29][30][31] begin to consider the influence of multiple poses. Most of them deal with the problem indirectly by random schemes or data augment, and they can only handle small changes in poses.…”

Robust facial landmark detection and tracking across poses and expressions for in-the-wild monocular video

“…Recently, discriminative models have shown promising performance for robust facial landmark detection, represented by cascaded regression-based methods, e.g., explicit shape regression [7], and the supervised descent method [8]. Many recent works following the cascaded regression framework consider how to improve efficiency [9,10] and accuracy, taking into account variations in pose, expression, lighting, and partial occlusion [11,12]. Although previous works have produced remarkable results on nearly frontal facial landmark detection, it is still not easy to locate landmarks across a large range of poses under uncontrolled conditions.…”