Robust Facial Landmark Detection via Occlusion-Adaptive Deep Networks

Zhu, Meilu; Shi, Daming; Zheng, Mingjie; Sadiq, Muhammad

doi:10.1109/cvpr.2019.00360

Cited by 111 publications

(69 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The proposed method is compared to other state-of-the-art landmark localization methods. From these approaches, coordinate regression methods include SDM [35], TSCN [26], IFA [1], CFSS [40], TCDCN [38], TSTN [14], DSRN [17], ODN [39], STA [30], Sun et al's work [27] and GAN [36]. Heatmap regression methods include Newell et al's work [18], SAN [9], LAB [32], CNN-CRF [5], LaplaceKL [23], Sun et al's work [28], DSNT [19] , DARK [37], FHR [30], GHCU [15] [10,11,21] for landmark detection are trained under different conditions with our method so are not included in our comparison.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Attentive One-Dimensional Heatmap Regression for Facial Landmark Detection and Tracking

Yin

Wang

Chen

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Although heatmap regression is considered a state-of-the-art method to locate facial landmarks, it suffers from huge spatial complexity and is prone to quantization error. To address this, we propose a novel attentive one-dimensional heatmap regression method for facial landmark localization. First, we predict two groups of 1D heatmaps to represent the marginal distributions of the and coordinates. These 1D heatmaps reduce spatial complexity significantly compared to current heatmap regression methods, which use 2D heatmaps to represent the joint distributions of and coordinates. With much lower spatial complexity, the proposed method can output high-resolution 1D heatmaps despite limited GPU memory, significantly alleviating the quantization error. Second, a co-attention mechanism is adopted to model the inherent spatial patterns existing in and coordinates, and therefore the joint distributions on the and axes are also captured. Third, based on the 1D heatmap structures, we propose a facial landmark detector capturing spatial patterns for landmark detection on an image; and a tracker further capturing temporal patterns with a temporal refinement mechanism for landmark tracking. Experimental results on four benchmark databases demonstrate the superiority of our method. CCS CONCEPTS • Computing methodologies → Biometrics.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Attentive One-Dimensional Heatmap Regression for Facial Landmark Detection and Tracking

Yin

Wang

Chen

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…Results on COFW: The comparison is shown in Table 3. Our method achieves similar performance compared to a recent method ODN (Zhu et al, 2019b). Note that our method is not specifically designed for occluded images (local boundary information may be obscured by occlusions).…”

Section: Results On Aflwmentioning

confidence: 66%

“…Common Challenge Full Inter-Pupil Distance NME (%) ESR (Cao et al, 2014) 5.28 17.00 7.58 SDM (Xiong and De la Torre, 2013) 5.57 15.40 7.52 LBF (Ren et al, 2014) 4.95 11.98 6.32 TCDCN (Zhang et al, 2014b) 4.80 8.60 5.54 CFSS (Zhu et al, 2015) 4.73 9.98 5.76 MDM (Trigeorgis et al, 2016) 4.83 10.14 5.88 Lv et al (2017) 4.36 7.56 4.99 AAN (Yue et al, 2018) 4.38 9.44 5.39 ECT 4.66 7.96 5.31 DSRN (Miao et al, 2018) 4.12 9.68 5. (Kumar and Chellappa, 2018) 3.67 7.62 4.44 JDR (Zhu et al, 2019a) 3.68 7.16 4.36 SAN (Dong et al, 2018a) 3.34 6.60 3.98 Reg + SBR (Dong et al, 2018b) 7.93 15.98 9.46 CPM + SBR (Dong et al, 2018b) 3.28 7.58 4.10 ODN (Zhu et al, 2019b) 3.56 6.67 4.17 ADN Sadiq et al (2019) 3 has inferior performance compared to the HRMs (CPM+SBR). By integrating our Fine-grained Facial Landmark Detection (FFLD) framework, our CRM based model has a comparable performance to state-of-the-art HRMs (Kumar and Chellappa, 2018;Dong et al, 2018b,a;Tai et al, 2019).…”

Section: Methodsmentioning

confidence: 99%

Fine-grained facial landmark detection exploiting intermediate feature representations

Yan

Duffner

Phutane

et al. 2020

Computer Vision and Image Understanding

View full text Add to dashboard Cite

Facial landmark detection has been an active research subject over the last decade. In this paper, we present a new approach for Fine-grained Facial Landmark Detection (FFLD) improving on the precision of the detected points. A high spatial precision of facial landmarks is crucial for many applications related to aesthetic rendering, such as face modeling, face animation, virtual make-up, etc. In this paper, we present an approach that improves the detection precision. Since most facial landmarks are positioned on visible boundary lines, we train a model that encourages the detected landmarks to stay on these boundaries. Our proposed Convolutional Neural Networks (CNN) effectively exploits lower-level feature maps containing abundant boundary information. To this end, beside the main CNN predicting facial landmark positions, we use several additional components, called CropNets. CropNet receives patches cropped from feature maps at different stages of this CNN, and estimate fine corrections of its predicted positions. We also introduce a novel robust spatial loss function based on pixel-wise differences between patches cropped from predicted and ground-truth positions. To further improve the landmark localisation, our framework uses several loss functions optimising the precision at several stages in different ways. Extensive experiments show that our framework significantly increases the local precision of state-of-the-art deep coordinate regression models.

show abstract

“…The model needed only an image as input and solved the problem of occlusion of large areas by detecting visible facial components while existing face detectors failed to detect faces. Zhu and colleagues developed Occlusion-adaptive Deep Networks (ODNs) in 2019 to solve the occlusion problem [21]. ODNs extract feature maps using the residual blocks, and then use the extracted feature maps as input into two CNN modules: A Geometry-aware Module and a Distillation Module.…”

Section: A Facial Landmark Detectionmentioning

confidence: 99%

FLSNet: Robust Facial Landmark Semantic Segmentation

2020

View full text Add to dashboard Cite

The human face is one of the most viewed visual objects in a person's life and is used for identifying a person through facial landmarks, which includes the eyes, nose, mouth, and ears that make up a face. It is also possible to communicate nonverbally through the movements of facial landmarks; that is, change of facial expression. Thus, facial landmarks play a crucial role in human-related image analysis. Automatic facial landmark detection is a challenging problem in the field of computer vision, and various studies are underway. The emergence of Deep Neural Networks has played an important role in solving difficult problems in computer vision. Semantic segmentation is a field in which images are classified into pixel units and has also developed rapidly by incorporating deep learning. In this paper, we propose a method for accurately extracting facial landmarks using semantic segmentation. First, we introduce a semantic segmentation architecture for sophisticated landmark detection, and datasets composed of facial images and ground truth pairs. Then, we suggest how improve the performance of pixel classification by adjusting the imbalance of the number of pixels according to the face landmark. Through extensive experiments, we evaluated our approach using the metrics pixel accuracy and intersection over union.

show abstract

Robust Facial Landmark Detection via Occlusion-Adaptive Deep Networks

Cited by 111 publications

References 50 publications

Attentive One-Dimensional Heatmap Regression for Facial Landmark Detection and Tracking

Attentive One-Dimensional Heatmap Regression for Facial Landmark Detection and Tracking

Fine-grained facial landmark detection exploiting intermediate feature representations

FLSNet: Robust Facial Landmark Semantic Segmentation

Contact Info

Product

Resources

About