A dual branch attention network based on practical degradation model for face super resolution

Abstract

In recent years, the field of face super-resolution (FSR) has advanced rapidly. However, complex degradation factors in real-world scenarios can severely deteriorate image quality, significantly affecting the reconstruction performance of FSR methods. Currently, there is a lack of research on degradation modeling for real-world facial images, which impacts the generalization ability of existing FSR methods. In this paper, a practical degradation model based on hybrid degradation processes is proposed to select multiple degradation processes including Gaussian noise, Rayleigh noise, Motion blur, Salt-and-Pepper noise, and Mean blur through a stochastic strategy to more realistically simulate the effect of image distortion in real scenarios. We also design a dual-branch attention network called DBANet for face super-resolution and conduct experiments on the SCUT_FBP, Helen and PFHQ datasets, achieving satisfactory results. Our proposed model is effective in handling image distortion under different degradation modalities, which improves the robustness of super-resolution reconstruction. This study introduces an innovative approach to the field of face image superresolution, which has the potential for a wide range of practical applications. The code of DBANet will be available at https://github.com/bxzha/DBANet.Face Super-Resolution (FSR) is a technique that reconstructs low-resolution facial images into high-resolution images through a series of algorithms. Due to the limitations of capture devices and transmission media, the quality of acquired facial images is often low, posing significant challenges for subsequent tasks such as face recognition and face segmentation. As early as 2000, Baker and Kanade 1 first proposed the concept of FSR, which achieved facial image restoration through statistical modeling and learning inference, thus initiating the wave of traditional FSR methods. However, due to the limited representation capabilities of traditional methods, they struggled to effectively reconstruct high-resolution and reliable facial images. In recent years, with the rapid development of deep learning and convolutional neural networks, FSR has been widely applied and has gradually attracted more attention from scholars in the field. Nevertheless, many existing FSR methods have not fully addressed the intrinsic characteristics of the super-resolution task.In a previous study 2 , a unique concept of "semantics" in super-resolution networks, the Deep Degradation Representation (DDR), has been demonstrated. This "semantic" concept is related to image degradation rather than image content.This means that the super-resolution network can learn to recognize specific degradation patterns from its training data distribution and has the capability to differentiate between various types of degradation. Different networks may learn distinct semantic representations, but when confronted with degraded inputs that fall outside their training distribution, their super-resolution performance tends to decline. By including mor...