RED-Net: A Recurrent Encoder–Decoder Network for Video-Based Face Alignment

Peng, Xi; Feris, Rogério; Wang, Xiaoyu; Metaxas, Dimitris N.

doi:10.1007/s11263-018-1095-1

Cited by 19 publications

(8 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To evaluate our method comprehensively and verify its effectiveness, we conducted extensive experiments and compared it with DIP and eight other state-of-the-art image denoising methods, including BM3D [2], NCSR [7], WNNM [5], DnCNN [11], FFDNet [12], TWSC [23], RED-Net [24], and CsNet [13]. We conducted denoising experiments on four datasets.…”

Section: Datasets and Experimental Setupmentioning

confidence: 99%

Image Denoising Using a Novel Deep Generative Network with Multiple Target Images and Adaptive Termination Condition

Chen

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

Image denoising, a classic ill-posed problem, aims to recover a latent image from a noisy measurement. Over the past few decades, a considerable number of denoising methods have been studied extensively. Among these methods, supervised deep convolutional networks have garnered increasing attention, and their superior performance is attributed to their capability to learn realistic image priors from a large amount of paired noisy and clean images. However, if the image to be denoised is significantly different from the training images, it could lead to inferior results, and the networks may even produce hallucinations by using inappropriate image priors to handle an unseen noisy image. Recently, deep image prior (DIP) was proposed, and it overcame this drawback to some extent. The structure of the DIP generator network is capable of capturing the low-level statistics of a natural image using an unsupervised method with no training images other than the image itself. Compared with a supervised denoising model, the unsupervised DIP is more flexible when processing image content that must be denoised. Nevertheless, the denoising performance of DIP is usually inferior to the current supervised learning-based methods using deep convolutional networks, and it is susceptible to the over-fitting problem. To solve these problems, we propose a novel deep generative network with multiple target images and an adaptive termination condition. Specifically, we utilized mainstream denoising methods to generate two clear target images to be used with the original noisy image, enabling better guidance during the convergence process and improving the convergence speed. Moreover, we adopted the noise level estimation (NLE) technique to set a more reasonable adaptive termination condition, which can effectively solve the problem of over-fitting. Extensive experiments demonstrated that, according to the denoising results, the proposed approach significantly outperforms the original DIP method in tests on different databases. Specifically, the average peak signal-to-noise ratio (PSNR) performance of our proposed method on four databases at different noise levels is increased by 1.90 to 4.86 dB compared to the original DIP method. Moreover, our method achieves superior performance against state-of-the-art methods in terms of popular metrics, which include the structural similarity index (SSIM) and feature similarity index measurement (FSIM). Thus, the proposed method lays a good foundation for subsequent image processing tasks, such as target detection and super-resolution.

show abstract

Section: Datasets and Experimental Setupmentioning

confidence: 99%

Image Denoising Using a Novel Deep Generative Network with Multiple Target Images and Adaptive Termination Condition

Chen

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…A joint optimization of alignment and tracking over a cascade with DNN-based features was proposed in Khan et al [ 48 ]. Finally, end-to-end learning via DNNs was also applied to video, e.g., using recurrent neural networks [ 20 , 27 ] and two-stream transformer networks [ 24 ].…”

Section: Related Workmentioning

confidence: 99%

“…At first, the detection of pose change activates the alignment, preventing unnecessary processing when the head doesn’t move. Alignment is then performed by regression cascade of tree ensembles, exploiting their superior computational efficiency [ 9 , 10 ] with respect to (possibly more accurate) state-of-the-art alignment methods based on deep neural networks (DNNs) [ 5 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 ]. In a scenario where energy efficiency is a matter of the utmost importance, a DNN-based alignment pre-processor might eclipse the energy efficiency advantage of ECs.…”

Section: Introductionmentioning

confidence: 99%

Face Pose Alignment with Event Cameras

Savran

Bartolozzi

2020

Sensors

View full text Add to dashboard Cite

Event camera (EC) emerges as a bio-inspired sensor which can be an alternative or complementary vision modality with the benefits of energy efficiency, high dynamic range, and high temporal resolution coupled with activity dependent sparse sensing. In this study we investigate with ECs the problem of face pose alignment, which is an essential pre-processing stage for facial processing pipelines. EC-based alignment can unlock all these benefits in facial applications, especially where motion and dynamics carry the most relevant information due to the temporal change event sensing. We specifically aim at efficient processing by developing a coarse alignment method to handle large pose variations in facial applications. For this purpose, we have prepared by multiple human annotations a dataset of extreme head rotations with varying motion intensity. We propose a motion detection based alignment approach in order to generate activity dependent pose-events that prevents unnecessary computations in the absence of pose change. The alignment is realized by cascaded regression of extremely randomized trees. Since EC sensors perform temporal differentiation, we characterize the performance of the alignment in terms of different levels of head movement speeds and face localization uncertainty ranges as well as face resolution and predictor complexity. Our method obtained 2.7% alignment failure on average, whereas annotator disagreement was 1%. The promising coarse alignment performance on EC sensor data together with a comprehensive analysis demonstrate the potential of ECs in facial applications.

show abstract

“…Sun et al [27] presented an approach for detecting the positions of face keypoints with three-level carefully designed convolution networks. In addition to convolutional neural networks(CNN), recurrent neural networks(RNN) have also been developed on feature point detection, especially on facial feature point detection [25], [28], [29].…”

Section: Related Workmentioning

confidence: 99%

High-Dimensional Data Set Simplification by Laplace-Beltrami Operator

Xu¹,

Lin²

2020

Preprint

View full text Add to dashboard Cite

With the development of the Internet and other digital technologies, the speed of data generation has become considerably faster than the speed of data processing. Because big data typically contain massive redundant information, it is possible to significantly simplify a big data set while maintaining the key information it contains. In this paper, we develop a big data simplification method based on the eigenvalues and eigenfunctions of the Laplace-Beltrami operator (LBO). Specifically, given a data set that can be considered as an unorganized data point set in high-dimensional space, a discrete LBO defined on the big data set is constructed and its eigenvalues and eigenvectors are calculated. Then, the local extremum and the saddle points of the eigenfunctions are proposed to be the feature points of a data set in high-dimensional space, constituting a simplified data set. Moreover, we develop feature point detection methods for the functions defined on an unorganized data point set in high-dimensional space, and devise metrics for measuring the fidelity of the simplified data set to the original set. Finally, examples and applications are demonstrated to validate the efficiency and effectiveness of the proposed methods, demonstrating that data set simplification is a method for processing a maximum-sized data set using a limited data processing capability.

show abstract

RED-Net: A Recurrent Encoder–Decoder Network for Video-Based Face Alignment

Cited by 19 publications

References 54 publications

Image Denoising Using a Novel Deep Generative Network with Multiple Target Images and Adaptive Termination Condition

Image Denoising Using a Novel Deep Generative Network with Multiple Target Images and Adaptive Termination Condition

Face Pose Alignment with Event Cameras

High-Dimensional Data Set Simplification by Laplace-Beltrami Operator

Contact Info

Product

Resources

About