Learning Spatial Attention for Face Super-Resolution

Chen, Chaofeng; Gong, Dihong; Wang, Hao; Li, Zhifeng; Wong, Kwan-Yee K.

doi:10.1109/tip.2020.3043093

Cited by 167 publications

(86 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• SPARNet [24], which uses a spatial attention mechanism to focus the generation process on key face structure regions.…”

Section: Comparison With Other Published Methodsmentioning

confidence: 99%

“…Further, a thresholdbased fusion and reconstruction module combines the candidate HR image to give the final SR prediction. Chen et al [24] proposed facial attention units(FAUs) which used a spatial attention mechanism to learn and focus on different face structures.…”

Section: Face Hallucinationmentioning

confidence: 99%

“…In the field of face hallucination, ATMFN [23] used an attention module to recognize the most relevant regions from all candidate SR images. SPARNet [24] used spatial attention in the generation process to progressively focus on various facial structures.…”

Section: Attention Networkmentioning

confidence: 99%

See 2 more Smart Citations

AGA-GAN: Attribute Guided Attention Generative Adversarial Network with U-Net for Face Hallucination

Srivastava¹,

Chanda²,

Pal³

2021

Preprint

View full text Add to dashboard Cite

The performance of facial super-resolution methods relies on their ability to recover facial structures and salient features effectively. Even though the convolutional neural network and generative adversarial network-based methods deliver impressive performances on face hallucination tasks, the ability to use attributes associated with the low-resolution images to improve performance is unsatisfactory. In this paper, we propose an Attribute Guided Attention Generative Adversarial Network which employs novel attribute guided attention (AGA) modules to identify and focus the generation process on various facial features in the image. Stacking multiple AGA modules enables the recovery of both high and low-level facial structures. We design the discriminator to learn discriminative features exploiting the relationship between the high-resolution image and their corresponding facial attribute annotations. We then explore the use of U-Net based architecture to refine existing predictions and synthesize further facial details. Extensive experiments across several metrics show that our AGA-GAN and AGA-GAN+U-Net framework outperforms several other cutting-edge face hallucination state-of-theart methods. We also demonstrate the viability of our method when every attribute descriptor is not known and thus, establishing its application in real-world scenarios.

show abstract

“…• SPARNet [24], which uses a spatial attention mechanism to focus the generation process on key face structure regions.…”

Section: Comparison With Other Published Methodsmentioning

confidence: 99%

Section: Face Hallucinationmentioning

confidence: 99%

See 1 more Smart Citation

AGA-GAN: Attribute Guided Attention Generative Adversarial Network with U-Net for Face Hallucination

Srivastava¹,

Chanda²,

Pal³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The generator tries to fool the discriminator by producing realistic fake images while the discriminator learns to distinguish real images from fake ones. GANs have been utilized to solve various images problems including image synthesis [14], image-to-image translation [15,16], image super-resolution [17,18], facial attribute editing [19,10,20] and expression manipulation [3,4,5].…”

Section: Related Workmentioning

confidence: 99%

US-GAN: On the importance of Ultimate Skip Connection for Facial Expression Synthesis

Akram¹,

Khan²

2021

Preprint

View full text Add to dashboard Cite

Recent studies have shown impressive results in multi-domain image-to-image translation for facial expression synthesis. While effective, these methods require a large number of labelled samples for plausible results. Their performance significantly degrades when we train them on smaller datasets. To address this limitation, in this work, we present US-GAN, a smaller and effective method for synthesizing plausible expressions by employing notably smaller datasets. The proposed method comprises of encoding layers, single residual block, decoding layers and an ultimate skip connection that links the input image to an output image. It has three times lesser parameters as compared to state-of-the-art facial expression synthesis methods. Experimental results demonstrate the quantitative and qualitative effectiveness of our proposed method. In addition, we also show that an ultimate skip connection is sufficient for recovering rich facial and overall color details of the input face image that a larger state-of-the-art model fails to recover.Keywords Expression synthesis • Image-to-image translation • GAN • FES.

show abstract

“…The single image super-resolution performance on benchmark datasets has been boosted continuously by numerous proposed models [1,2,3,4,25], which usually achieve state-of-the-art performance via advanced network architecture or learning strategy. Some fundamental and classic network designs in SR models contain recursive learning [26], residual learning [27], dense connection [28], multipath learning [29] and attention mechanism [2,30,31]. Kim et al [26] proposed Deeply-recursive Convolutional Network (DRCN) to recursively learning high-level representations with a weight-shared module.…”

Section: Related Workmentioning

confidence: 99%

SelFSR: Self-Conditioned Face Super-Resolution in the Wild via Flow Field Degradation Network

Zeng¹,

Zhang²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

In spite of the success on benchmark datasets, most advanced face super-resolution models perform poorly in real scenarios since the remarkable domain gap between the real images and the synthesized training pairs. To tackle this problem, we propose a novel domain-adaptive degradation network for face super-resolution in the wild. This degradation network predicts a flow field along with an intermediate low resolution image. Then, the degraded counterpart is generated by warping the intermediate image. With the preference of capturing motion blur, such a model performs better at preserving identity consistency between the original images and the degraded. We further present the self-conditioned block for super-resolution network. This block takes the input image as a condition term to effectively utilize facial structure information, eliminating the reliance on explicit priors, e.g. facial landmarks or boundary. Our model achieves state-of-the-art performance on both CelebA and real-world face dataset. The former demonstrates the powerful generative ability of our proposed architecture while the latter shows great identity consistency and perceptual quality in realworld images.

show abstract

Learning Spatial Attention for Face Super-Resolution

Cited by 167 publications

References 49 publications

AGA-GAN: Attribute Guided Attention Generative Adversarial Network with U-Net for Face Hallucination

AGA-GAN: Attribute Guided Attention Generative Adversarial Network with U-Net for Face Hallucination

US-GAN: On the importance of Ultimate Skip Connection for Facial Expression Synthesis

SelFSR: Self-Conditioned Face Super-Resolution in the Wild via Flow Field Degradation Network

Contact Info

Product

Resources

About