SRFeat: Single Image Super-Resolution with Feature Discrimination

Park, Seong-Jin; Son, Hyeongseok; Cho, Sunghyun; Hong, Ki-Sang; Lee, Seungyong

doi:10.1007/978-3-030-01270-0_27

Cited by 166 publications

(100 citation statements)

References 42 publications

Supporting

Mentioning

100

Contrasting

Order By: Relevance

“…Finally we directly calculate the adversarial loss in this feature map, since receptive fields of each point in this feature map can still cover the entire input image. Our feature patch discriminator combines the advantages of the conventional feature discriminator [29] and patch discriminator [18], which is not only fast and stable during training but also makes the refinement network synthesize more meaningful highfrequency details.…”

Section: Feature Patch Discriminatormentioning

confidence: 99%

“…Effect of feature patch discriminator As shown in Fig 11(b), when we only use the patch discriminator, the result performances distorted structure. Then we add the conventional feature discriminator [29], however the generated content still seems blurry (See Fig 11(c)). Finally, by performing the feature patch discriminator, fine details and reasonable structure can be obtained (See Fig 11(d)).…”

Section: Ablation Studymentioning

confidence: 99%

“…Finally, by performing the feature patch discriminator, fine details and reasonable structure can be obtained (See Fig 11(d)). Moreover, the feature patch discriminator processes each image for 0.2 seconds faster than the conventional one [29]. Figure 11.…”

Section: Ablation Studymentioning

confidence: 99%

“…In order to make network training process more stable and motivate the CSA layer to learn more effective features, we propose a consistency loss to measure not only the distance between the VGG feature layer and the CSA layer but also the distance between the VGG feature layer and the the corresponding layer of the CSA in decoder. Meanwhile, in addition to a patch discriminator [18], we improve the details by introducing a feature patch which is simpler in formulation, faster and more stable for training than conventional one [29]. Except for the consistency loss, reconstruction loss, and relativistic average LS adversarial loss [28] are incorporated as constraints to instruct our model to learn meaningful parameters.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Coherent Semantic Attention for Image Inpainting

Liu

Jiang

Xiao

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

346

200

View full text Add to dashboard Cite

The latest deep learning-based approaches have shown promising results for the challenging task of inpainting missing regions of an image. However, the existing methods often generate contents with blurry textures and distorted structures due to the discontinuity of the local pixels. From a semantic-level perspective, the local pixel discontinuity is mainly because these methods ignore the semantic relevance and feature continuity of hole regions. To handle this problem, we investigate the human behavior in repairing pictures and propose a fined deep generative model-based approach with a novel coherent semantic attention (CSA) layer, which can not only preserve contextual structure but also make more effective predictions of missing parts by modeling the semantic relevance between the holes features. The task is divided into rough, refinement as two steps and model each step with a neural network under the U-Net architecture, where the CSA layer is embedded into the encoder of refinement step. To stabilize the network training process and promote the CSA layer to learn more effective parameters, we propose a consistency loss to enforce the both the CSA layer and the corresponding layer of the CSA in decoder to be close to the VGG feature layer of a ground truth image simultaneously. The experiments on CelebA, Places2, and Paris StreetView datasets have validated the effectiveness of our proposed methods in image inpainting tasks and can obtain images with a higher quality as compared with the existing state-of-the-art approaches. The codes and pretrained models will be available at https://github. com/KumapowerLIU/CSA-inpainting.

show abstract

Section: Feature Patch Discriminatormentioning

confidence: 99%

Section: Ablation Studymentioning

confidence: 99%

Section: Ablation Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Coherent Semantic Attention for Image Inpainting

Liu

Jiang

Xiao

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

346

200

View full text Add to dashboard Cite

show abstract

“…The goal of single-image SR is to fill in HR image patterns by leveraging a prior derived either from similar patches in other parts of the input image (self-examplars) [14,21], or from similar image patches from an existing image database [13], or -most commonly -from previously seen training data [25,48,53,54]. Recently the technologies of choice for learning the prior have been (deep) convolutional networks [10,64,46,22,66,49] and generative adversarial networks [31,60,42,61,59]. An overview of recent singleimage SR methods is given in the report of the NTIRE Challenge [3].…”

Section: Related Workmentioning

confidence: 99%

Learned Multi-View Texture Super-Resolution

Richard

Cherabier

Oswald

et al. 2019

2019 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

We present a super-resolution method capable of creating a high-resolution texture map for a virtual 3D object from a set of lower-resolution images of that object. Our architecture unifies the concepts of (i) multi-view superresolution based on the redundancy of overlapping views and (ii) single-view super-resolution based on a learned prior of high-resolution (HR) image structure. The principle of multi-view super-resolution is to invert the image formation process and recover the latent HR texture from multiple lower-resolution projections. We map that inverse problem into a block of suitably designed neural network layers, and combine it with a standard encoder-decoder network for learned single-image super-resolution. Wiring the image formation model into the network avoids having to learn perspective mapping from textures to images, and elegantly handles a varying number of input views. Experiments demonstrate that the combination of multi-view observations and learned prior yields improved texture maps.

show abstract

Spatial orthogonal attention generative adversarial network for MRI reconstruction

Zhou

Mei

et al. 2020

Medical Physics

View full text Add to dashboard Cite

Purpose: Recent studies have witnessed that self-attention modules can better solve the vision understanding problems by capturing long-range dependencies. However, there are very few works designing a lightweight self-attention module to improve the quality of MRI reconstruction. Furthermore, it can be observed that several important self-attention modules (e.g., the non-local block) cause high computational complexity and need a huge number of GPU memory when the size of the input feature is large. The purpose of this study is to design a lightweight yet effective spatial orthogonal attention module (SOAM) to capture long-range dependencies, and develop a novel spatial orthogonal attention generative adversarial network, termed as SOGAN, to achieve more accurate MRI reconstruction. Methods: We first develop a lightweight SOAM, which can generate two small attention maps to effectively aggregate the long-range contextual information in vertical and horizontal directions, respectively. Then, we embed the proposed SOAMs into the concatenated convolutional autoencoders to form the generator of the proposed SOGAN. Results: The experimental results demonstrate that the proposed SOAMs improve the quality of the reconstructed MR images effectively by capturing long-range dependencies. Besides, compared with state-of-the-art deep learning-based CS-MRI methods, the proposed SOGAN reconstructs MR images more accurately, but with fewer model parameters. Conclusions: The proposed SOAM is a lightweight yet effective self-attention module to capture long-range dependencies, thus, can improve the quality of MRI reconstruction to a large extent. Besides, with the help of SOAMs, the proposed SOGAN outperforms the state-of-the-art deep learning-based CS-MRI methods.

show abstract

SRFeat: Single Image Super-Resolution with Feature Discrimination

Cited by 166 publications

References 42 publications

Coherent Semantic Attention for Image Inpainting

Coherent Semantic Attention for Image Inpainting

Learned Multi-View Texture Super-Resolution

Spatial orthogonal attention generative adversarial network for MRI reconstruction

Contact Info

Product

Resources

About