DeepSEE: Deep Disentangled Semantic Explorative Extreme Super-Resolution

Bühler, Marcel C.; Romero, Andrés; Timofte, Radu

doi:10.1007/978-3-030-69538-5_38

Cited by 21 publications

(10 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Stochastic Super Resolution: Recently, there has been a growing body of work dealing with the creation of multiple possible solutions for a given super resolution (SR) problem [37,18,38]. These often stochastic methods typically produce sharper images, with finer details, than their deterministic counterparts which can be attributed to the extreme ill-posedness of the SR problem.…”

Section: Related Workmentioning

confidence: 99%

Posterior Sampling for Image Restoration using Explicit Patch Priors

Friedman,

Weiss

2021

Preprint

View full text Add to dashboard Cite

Almost all existing methods for image restoration are based on optimizing the mean squared error (MSE), even though it is known that the best estimate in terms of MSE may yield a highly atypical image due to the fact that there are many plausible restorations for a given noisy image. In this paper, we show how to combine explicit priors on patches of natural images in order to sample from the posterior probability of a full image given a degraded image. We prove that our algorithm generates correct samples from the distribution p(x|y) ∝ exp(−E(x|y)) where E(x|y) is the cost function minimized in previous patch-based approaches that compute a single restoration. Unlike previous approaches that computed a single restoration using MAP or MMSE, our method makes explicit the uncertainty in the restored images and guarantees that all patches in the restored images will be typical given the patch prior. Unlike previous approaches that used implicit priors on fixed-size images, our approach can be used with images of any size. Our experimental results show that posterior sampling using patch priors yields images of high perceptual quality and high PSNR on a range of challenging image restoration problems.

show abstract

Section: Related Workmentioning

confidence: 99%

Posterior Sampling for Image Restoration using Explicit Patch Priors

Friedman,

Weiss

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Most recently, Lugmayr et al [29] proposed to use conditional normalizing flow to model image distribution for super-resolution and achieved good performance in both quantitative and qualitative results. Normalizing flow based approaches can also be extended to image manipulation [4,7]. However, the whole normalizing flow structure is over-complex (approx.…”

Section: Generative Learning Approaches For Srmentioning

confidence: 99%

Variational AutoEncoder for Reference based Image Super-Resolution

Liu

Siu

Wang

2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

In this paper, we propose a novel reference based image super-resolution approach via Variational AutoEncoder (RefVAE). Existing state-of-the-art methods mainly focus on single image super-resolution which cannot perform well on large upsampling factors, e.g., 8×. We propose a reference based image super-resolution, for which any arbitrary image can act as a reference for super-resolution. Even using random map or low-resolution image itself, the proposed RefVAE can transfer the knowledge from the reference to the super-resolved images. Depending upon different references, the proposed method can generate different versions of super-resolved images from a hidden superresolution space. Besides using different datasets for some standard evaluations with PSNR and SSIM, we also took part in the NTIRE2021 SR Space challenge [28] and have provided results of the randomness evaluation of our approach. Compared to other state-of-the-art methods, our approach achieves higher diverse scores.

show abstract

“…Recently, Image-to-image (I2I) translation has become a very active topic thanks to the impressive advances in generative modeling methods, and in particular, Generative Adversarial Networks (GANs) [10]. Several novel and challenging problems have been successfully tackled with this technique, e.g., multi-domain manipulation [8,33], style transferring [14,24], image inpainting [44,29], image synthesis using semantic segmentation [30,50,22], image content manipulation [31], exploratory image superresolution [27,5], etc.…”

Section: Related Workmentioning

confidence: 99%

SMILE: Semantically-guided Multi-attribute Image and Layout Editing

Romero¹,

Gool²,

Timofte³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Attribute image manipulation has been a very active topic since the introduction of Generative Adversarial Networks (GANs). Exploring the disentangled attribute space within a transformation is a very challenging task due to the multiple and mutually-inclusive nature of the facial images, where different labels (eyeglasses, hats, hair, identity, etc.) can co-exist at the same time. Several works address this issue either by exploiting the modality of each domain/attribute using a conditional random vector noise, or extracting the modality from an exemplary image. However, existing methods cannot handle both random and reference transformations for multiple attributes, which limits the generality of the solutions. In this paper, we successfully exploit a multimodal representation that handles all attributes, be it guided by random noise or exemplar images, while only using the underlying domain information of the target domain. We present extensive qualitative and quantitative results for facial datasets and several different attributes that show the superiority of our method. Additionally, our method is capable of adding, removing or changing either fine-grained or coarse attributes by using an image as a reference or by exploring the style distribution space, and it can be easily extended to head-swapping and face-reenactment applications without being trained on videos.

show abstract

DeepSEE: Deep Disentangled Semantic Explorative Extreme Super-Resolution

Cited by 21 publications

References 56 publications

Posterior Sampling for Image Restoration using Explicit Patch Priors

Posterior Sampling for Image Restoration using Explicit Patch Priors

Variational AutoEncoder for Reference based Image Super-Resolution

SMILE: Semantically-guided Multi-attribute Image and Layout Editing

Contact Info

Product

Resources

About