Deep Semantic Face Deblurring

Abstract

In this paper, we present an effective and efficient face deblurring algorithm by exploiting semantic cues via deep convolutional neural networks (CNNs). As face images are highly structured and share several key semantic components (e.g., eyes and mouths), the semantic information of a face provides a strong prior for restoration. As such, we propose to incorporate global semantic priors as input and impose local structure losses to regularize the output within a multi-scale deep CNN. We train the network with perceptual and adversarial losses to generate photo-realistic results and develop an incremental training strategy to handle random blur kernels in the wild. Quantitative and qualitative evaluations demonstrate that the proposed face deblurring algorithm restores sharp images with more facial details and performs favorably against state-of-the-art methods in terms of restoration quality, face recognition and execution speed.

Unsupervised Domain-Specific Deblurring via Disentangled Representations

“…Nah et al [26] produce good results on the first two faces, but generate some artifacts in the third image. Deep semantic face deblurring [36] generate better results than other compared methods. Nonetheless, due to the existence of face parsing, they tend to sharpen some facial parts (eye, nose and mouth) but over-smooth the ears and the background.…”

“…Moreover, we achieve comparable performance compared to other state-of-the-art supervised deblurring methods. Shen et al [36] perform very well for frontal-to-frontal protocol, yet provide the worst performance on frontal-toprofile protocol, which shows that the face parsing network in their method is sensitive to poses. In contrast, the proposed method works for both frontal and profile face images even though we do not explicitly train on faces with extreme poses.…”

“…Among them, Krishnan et al [14] perform the best in PSNR and SSIM and Pan et al [29] perform the best in perceptual distance. For CNN-based methods, Shen et al [36] include a face parsing branch and achieve the best performance among the compared methods. The results for DeblurGAN [15] contain some ringing artifacts and Cycle-GAN [44] cannot recover the mouth part of both images that well.…”

“…Pan et al [27] utilize exemplar faces in a reference set to guide the blur kernel estimation. Shen et al [36] use the face parsing labels as global semantic priors and local structure regularization to improve face deblurring performance.…”

FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videos

“…Prior knowledge, especially facial structure [15,2,35], has been proven to be an effective face prior in corresponding tasks such as super-resolution and deblurring. However, as mentioned in [35], these methods fail when the input face images are not well aligned, e.g. side faces or extremely large motion where semantic face parsing or landmark detection fails.…”