Rectified Wing Loss for Efficient and Robust Facial Landmark Localisation with Convolutional Neural Networks

Abstract

Efficient and robust facial landmark localisation is crucial for the deployment of real-time face analysis systems. This paper presents a new loss function, namely Rectified Wing (RWing) loss, for regression-based facial landmark localisation with Convolutional Neural Networks (CNNs). We first systemically analyse different loss functions, including L2, L1 and smooth L1. The analysis suggests that the training of a network should pay more attention to small-medium errors. Motivated by this finding, we design a piece-wise loss that amplifies the impact of the samples with small-medium errors. Besides, we rectify the loss function for very small errors to mitigate the impact of inaccuracy of manual annotation. The use of our RWing loss boosts the performance significantly for regression-based CNNs in facial landmarking, especially for lightweight network architectures. To address the problem of under-representation of samples with large pose variations, we propose a simple but effective boosting strategy, referred to as pose-based data balancing. In particular, we deal with the data imbalance problem by duplicating the minority training samples and perturbing them by injecting random image rotation, bounding box translation and other data augmentation strategies. Last, the proposed approach is extended to create a coarse-to-fine framework for robust and efficient landmark localisation. Moreover, the proposed coarse-to-fine framework is able to deal with the small sample size problem effectively. The experimental results obtained on several well-known benchmarking datasets demonstrate the merits of our RWing loss and prove the superiority of the proposed method over the state-of-the-art approaches.

Improving Face Alignment Accuracy on Clinical Populations and its effect on the Video-based Detection of Neurological Diseases

“…Optimization algorithm and hyper-parameters were the same to those used by Bulat and Tzimiropoulos to train the FAN model [35]. Our training algorithm used a recently introduced loss function, adaptive wing-loss, which improves model performance by penalizing small errors more than the traditional squared-loss [36], [52], [53] Twelve participants from each group were randomly selected and used to train the model. Data from the remaining participants were used to test the model performance by computing the accuracy in landmark localization.…”

“…These databases often consist of thousands of photographs with a large variety of poses, expressions, illumination, backgrounds, and scales. Thus, pre-trained FA models are designed to provide accurate facial landmark localization under general conditions [31]- [36].…”

Robust Face Alignment via Deep Progressive Reinitialization and Adaptive Error-driven Learning

“…Different Learning Objectives Several variations of the ℓ 1 or ℓ 2 based loss functions designed for face alignment have exhibited excellent accuracy. Wing [35] and RWing [54] both focus on small range errors and switch the loss function from an ℓ 1 loss to a modified logarithm function. AWing [55] applies a similar idea to improve the quality of heat map regression results.…”

Resolution Invariant Face Recognition Using a Distillation Approach

“…This alignment is carried out by first extracting facial landmarks and then performing an affine transformation based on the coordinates of five facial landmarks, i.e., eye centres, nose tip and mouth corners. There are many existing facial landmark detection algorithms, such as cascaded shape regression [41], [42] and CNN based methods [43]- [46]. However, standard facial landmark detection approaches such as MTCNN [43] do not work well for LR images, i.e., they fail to detect a large number of LR faces during training as well as testing.…”