Unsupervised Real-world Image Super Resolution via Domain-distance Aware Training

Wei, Yunxuan; Gu, Shuhang; Li, Yawei; Timofte, Radu; Jin, Longcun; Song, Hengjie

doi:10.1109/cvpr46437.2021.01318

Cited by 117 publications

(84 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general, they model complex real-world degradations in either an implicit or an explicit way. Between them, implicit methods [10,30,39,46] aim to learn a degradation network from real-world LR images. In the absence of corresponding ground truth HR images, most of them employed unsupervised image-to-image translation (e.g.…”

Section: Related Workmentioning

confidence: 99%

Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors

Chen¹,

Shi²,

Qin³

et al. 2022

Preprint

View full text Add to dashboard Cite

A key challenge of blind image super resolution is to recover realistic textures for low-resolution images with unknown degradations. Most recent works completely rely on the generative ability of GANs, which are difficult to train. Other methods resort to high-resolution image references that are usually not available. In this work, we propose a novel framework, denoted as QuanTexSR, to restore realistic textures with the Quantized Texture Priors encoded in Vector Quantized GAN. The QuanTexSR generates textures by aligning the textureless content features to the quantized feature vectors, i.e., a pretrained feature codebook. Specifically, QuanTexSR formulates the texture generation as a feature matching problem between textureless features and a pretrained feature codebook. The final textures are then generated by the quantized features from the codebook. Since features in the codebook have shown the ability to generate natural textures in the pretrain stage, QuanTexSR can generate rich and realistic textures with the pretrained codebook as texture priors. Moreover, we propose a semantic regularization technique that regularizes the pre-training of the codebook using clusters of features extracted from the pretrained VGG19 network. This further improves texture generation with semantic context. Experiments demonstrate that the proposed QuanTexSR can generate competitive or better textures than previous approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors

Chen¹,

Shi²,

Qin³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…One line of research employs generative adversarial networks (GANs) [13]. To learn from unpaired data, either cycle-consistency losses [24,7] or domain-based adversarial losses [12,34,6] are employed. Yet, these approaches suffer from convergence and mode collapse issues, requiring elaborate fine-tuning of their losses.…”

Section: Related Workmentioning

confidence: 99%

“…DASR [34] Frequency Separation [12] Impressionism [17] DeFlow (ours) LR White Noise CycleGAN [26] Frequency Separation † [12] Impressionism [17] DeFlow (ours)…”

Section: Lr White Noisementioning

confidence: 99%

“…Several approaches resort to hand-crafted strategies tailored to specific types of degradations [17]. Existing learnable solutions mostly adopt generative adversarial networks (GANs) with cycle-consistency constraints [39,24,7] or domainaware adversarial objectives [12,34,6] for unpaired training. However, these approaches require careful tuning of several losses.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows

Wolf¹,

Lugmayr²,

Danelljan³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The difficulty of obtaining paired data remains a major bottleneck for learning image restoration and enhancement models for real-world applications. Current strategies aim to synthesize realistic training data by modeling noise and degradations that appear in real-world settings. We propose DeFlow, a method for learning stochastic image degradations from unpaired data. Our approach is based on a novel unpaired learning formulation for conditional normalizing flows. We model the degradation process in the latent space of a shared flow encoder-decoder network. This allows us to learn the conditional distribution of a noisy image given the clean input by solely minimizing the negative log-likelihood of the marginal distributions. We validate our DeFlow formulation on the task of joint image restoration and super-resolution. The models trained with the synthetic data generated by DeFlow outperform previous learnable approaches on all three evaluated datasets.

show abstract

“…Single image super-resolution (SISR) is a classical computer vision problem that tries to infer a high-resolution (HR) image from a single low-resolution (LR) input image. This problem is still an active research field in the computer vision community (e.g., [ 1 , 2 , 3 , 4 ]). Several applications in different fields can benefit from super-resolution (SR) representations, for instance, security (e.g., [ 5 , 6 ]), medical imaging (e.g., [ 7 ]), object detection (e.g., [ 8 ]), and astronomical images (e.g., [ 9 ]), among others.…”

Section: Introductionmentioning

confidence: 99%

A Novel Domain Transfer-Based Approach for Unsupervised Thermal Image Super-Resolution

Rivadeneira

Sappa

Vintimilla

et al. 2022

Sensors

View full text Add to dashboard Cite

This paper presents a transfer domain strategy to tackle the limitations of low-resolution thermal sensors and generate higher-resolution images of reasonable quality. The proposed technique employs a CycleGAN architecture and uses a ResNet as an encoder in the generator along with an attention module and a novel loss function. The network is trained on a multi-resolution thermal image dataset acquired with three different thermal sensors. Results report better performance benchmarking results on the 2nd CVPR-PBVS-2021 thermal image super-resolution challenge than state-of-the-art methods. The code of this work is available online.

show abstract

Unsupervised Real-world Image Super Resolution via Domain-distance Aware Training

Cited by 117 publications

References 39 publications

Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors

Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors

DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows

A Novel Domain Transfer-Based Approach for Unsupervised Thermal Image Super-Resolution

Contact Info

Product

Resources

About