Robust Face Alignment by Multi-Order High-Precision Hourglass Network

Abstract

Heatmap regression (HR) has become one of the mainstream approaches for face alignment and has obtained promising results under constrained environments. However, when a face image suffers from large pose variations, heavy occlusions and complicated illuminations, the performances of HR methods degrade greatly due to the low resolutions of the generated landmark heatmaps and the exclusion of important high-order information that can be used to learn more discriminative features. To address the alignment problem for faces with extremely large poses and heavy occlusions, this paper proposes a heatmap subpixel regression (HSR) method and a multi-order cross geometry-aware (MCG) model, which are seamlessly integrated into a novel multi-order high-precision hourglass network (MHHN). The HSR method is proposed to achieve high-precision landmark detection by a well-designed subpixel detection loss (SDL) and subpixel detection technology (SDT). At the same time, the MCG model is able to use the proposed multi-order cross information to learn more discriminative representations for enhancing facial geometric constraints and context information. To the best of our knowledge, this is the first study to explore heatmap subpixel regression for robust and high-precision face alignment. The experimental results from challenging benchmark datasets demonstrate that our approach outperforms state-of-theart methods in the literature.

Subpixel Heatmap Regression for Facial Landmark Localization

“…However, despite their wide adoption, heatmap-based regression approaches suffer from discretization-induced errors. Although this is in general known, there are very few papers that study this problem[29,44,47]. Yet, in this paper, we show that this overlooked problem makes actually has surprisingly negative impact on the accuracy of the model.In particular, as working in high resolutions is computationally and memory prohibitive, typically, heatmap regression networks make predictions at1 4 of the input resolution[5].…”

“…NME ic (%) AUC 10 ic FR 10 ic (%) Wing [16] 5.11 0.554 6.00 MHHN [47] 4.77 -DeCaFa [10] 4.62 0.563 4.84 AVS [34] 4.39 0.591 4.08 AWing [49] 4.36 0.572 2.84 LUVLi [24] 4.37 0.577 3.12 GCN [26] 4 for other methods taken from [43].…”

“…In this direction, [44] proposes an analytic solution to obtain the fractional shift by assuming that the generated heatmap follows a Gaussian distribution and applies this to stabilize facial landmark localization in video. A similar assumption is made by [47] which solves an optimization problem to obtain the subpixel solution. Finally, [29] uses global softargmax.…”

“…In addition to the comparisons reported in the main paper here in we show how our method performs against an additional set of methods ( Tables 10,11,12,12,13). [62] 3.92 2.68 --CCL [63] 2.72 2.17 --DAC-CSR [15] 2.27 1.81 --LLL [36] 1.97 ---CPM+SRB [13] 2.14 ---SAN [12] 1.91 1.85 4.04 54.0 DSNR [30] 1.85 11.20 36.0 RCPR [7] 8.50 20.00 HPM 7.59 13.00 CCR [14] 7.03 10.90 DRDA [59] 6.49 6.00 RAR [55] 6.03 4.14 DAC-CSR [15] 6.03 4.73 LAB (w/o B) [51] 5.58 2.76 Wing [16] 5.07 3.16 MHHN [47] 4.95 1.78 LAB (w/B) [51] 3.92 0.39 HR-Net [43] 3.45 0.19 Ours 3.02 0.0…”

JDSR-GAN: Constructing An Efficient Joint Learning Network for Masked Face Super-Resolution

“…With the improvement awareness of epidemic prevention, face images captured in conventional unlimited scenes such as video surveillance possess complex variations such as mask and low-resolution (LR) simultaneously. Obtaining high-resolution (HR) face images without the mask is now an essential yet challenging task, which plays an import role in many face-related security applications, e.g., face alignment [1], face parsing [2], face detection [3], face tracking [4], and face recognition [5,6]. Although many existing approaches have achieved promising progress in attaining high-quality HR face samples from the related low-quality LR ones, most of them can only be used to handle one type of variation, such as LR face super-resolution or masked face image completion.…”