NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results

Lugmayr, Andreas; Danelljan, Martin; Timofte, Radu; Ahn, Namhyuk; Bai, Dingqun; Cai, Jie; Cao, Yun; Chen, Junyang; Cheng, Kaihua; Chun, SeYoung; Deng, Wei; El-Khamy, Mostafa; Ho, Chiu Man; Ji, Xiaozhong; Kheradmand, Amin; Kim, Gwantae; Ko, Hanseok; Lee, Kanghyu; Lee, Jung-won; Li, Hao; Liu, Ziluan; Liu, Zhisong; Liu, Shuai; Lu, Yunhua; Meng, Zibo; Michelini, Pablo Navarrete; Micheloni, Christian; Prajapati, Kalpesh; Ren, Haoyu; Seo, Yong Hyeok; Siu, Wan-Chi; Sohn, Kyung-Ah; Tai, Ying; Umer, Rao Muhammad; Wang, Shuangquan; Wang, Huibing; Wu, Timothy Haoning; Wu, Haoning; Biao, Yang; Yang, Fuzhi; Yoo, Jaejun; Zhao, Tongtong; Zhou, Yuanbo; Zhuo, Hui; Zong, Ziyao; Zou, Xueyi

doi:10.1109/cvprw50498.2020.00255

Cited by 157 publications

(134 citation statements)

References 56 publications

Supporting

Mentioning

132

Contrasting

Order By: Relevance

“…The works exploiting DL for SR are extensive, but they are primarily concerned with artificially generated LR images lacking any noise, and only recently the interest in real realworld image SR increased [18]. Practically, noisy images are typically considered for denoising tasks [19].…”

Section: Noise Modelling In Pclementioning

confidence: 99%

Zero-Shot Super-Resolution With a Physically-Motivated Downsampling Kernel for Endomicroscopy

Szczotka

Shakir

Clarkson

et al. 2021

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Super-resolution (SR) methods have seen significant advances thanks to the development of convolutional neural networks (CNNs). CNNs have been successfully employed to improve the quality of endomicroscopy imaging. Yet, the inherent limitation of research on SR in endomicroscopy remains the lack of ground truth highresolution (HR) images, commonly used for both supervised training and reference-based image quality assessment (IQA). Therefore, alternative methods, such as unsupervised SR are being explored. To address the need for non-reference image quality improvement, we designed a novel zero-shot super-resolution (ZSSR) approach that relies only on the endomicroscopy data to be processed in a self-supervised manner without the need for groundtruth HR images. We tailored the proposed pipeline to the idiosyncrasies of endomicroscopy by introducing both: a physically-motivated Voronoi downscaling kernel accounting for the endomicroscope's irregular fibre-based sampling pattern, and realistic noise patterns. We also took advantage of video sequences to exploit a sequence of images for self-supervised zero-shot image quality improvement. We run ablation studies to assess our contribution in regards to the downscaling kernel and noise simulation. We validate our methodology on both synthetic and original data. Synthetic experiments were assessed with reference-based IQA, while our results for original images were evaluated in a user study conducted with both expert and non-expert observers. The results demonstrated superior performance in image quality of ZSSR reconstructions in comparison to the baseline method. The ZSSR is also competitive when compared to supervised single-image SR, especially being the preferred reconstruction technique by experts.

show abstract

Section: Noise Modelling In Pclementioning

confidence: 99%

Zero-Shot Super-Resolution With a Physically-Motivated Downsampling Kernel for Endomicroscopy

Szczotka

Shakir

Clarkson

et al. 2021

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…In the next experiment, we applied the SRDNN methods (VDSR, EDSR, RRDB, ESRGAN, RCAN, and SAN) to the images of the MICC logo database [21]. Figure 24 shows an original JPG image, which was enlarged (4x) by using the SRDNN methods.…”

Section: Enhancing Coding Artefactsmentioning

confidence: 99%

“…These SRDNN methods aim to improve perceptual image quality instead of the conventional PSNR. Furthermore, applying SRDNN methods to real-world images has been investigated by assuming that paired HR (high-resolution) and LR images are unavailable [21][22][23]. Most existing SR (super-resolution) methods use degradation models that are not related to real images.…”

Section: Introductionmentioning

confidence: 99%

One-to-One Mapping-Like Properties of DCN-Based Super-Resolution and its Applicability to Real-World Images

et al. 2021

View full text Add to dashboard Cite

Although super-resolution techniques based on deep neural networks (SRDNN) have drawn significant interest and numerous algorithms have been proposed, they still have reliability problems and produce artefacts when applied to new datasets. In this paper, the working mechanisms of SRDNN techniques are analyzed in terms of data mapping. Since most SRDNN techniques can be viewed as dynamic linear projections, we analyzed a large number of projection vectors (over 70 million) and found that the SRDNN method performs one-to-one mapping-like operations and may be vulnerable to unknown data patterns. Then, we applied several SRDNN techniques to real-world images and analyzed the output images. The current SRDNN methods failed to distinguish the blurred edges/lines due to low resolutions from coding artefacts and enhanced both, even though the SRDNN methods were trained using compressed low-resolution (LR) images. These analyses and results indicate that current SRDNN methods may not be able to provide robust performance and new structures may be necessary for reliable super-resolution performance.

show abstract

“…It can digest extensive training data and extract discriminative representations with the support of powerful computational resource. CNN has shown significant advantages in various tasks, like image classification [18,31], semantic segmentation [28,39], super-resolution [9,24], place recognition [1,20], etc. CNNs with the deep structure are difficult to train because parameters of the shallow layers are often under gradient vanishing and exploding risks.…”

Section: Introductionmentioning

confidence: 99%

Deep Relighting Networks for Image Light Source Manipulation

Wang

Siu

Liu

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Manipulating the light source of given images is an interesting task and useful in various applications, including photography and cinematography. Existing methods usually require additional information like the geometric structure of the scene, which may not be available for most images. In this paper, we formulate the single image relighting task and propose a novel Deep Relighting Network (DRN) with three parts: 1) scene reconversion, which aims to reveal the primary scene structure through a deep auto-encoder network, 2) shadow prior estimation, to predict light effect from the new light direction through adversarial learning, and 3) re-renderer, to combine the primary structure with the reconstructed shadow view to form the required estimation under the target light source. Experiments show that the proposed method outperforms other possible methods, both qualitatively and quantitatively. Specifically, the proposed DRN has achieved the best PSNR in the "AIM2020 -Any to one relighting challenge" of the 2020 ECCV conference.

show abstract

NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results

Cited by 157 publications

References 56 publications

Zero-Shot Super-Resolution With a Physically-Motivated Downsampling Kernel for Endomicroscopy

Zero-Shot Super-Resolution With a Physically-Motivated Downsampling Kernel for Endomicroscopy

One-to-One Mapping-Like Properties of DCN-Based Super-Resolution and its Applicability to Real-World Images

Deep Relighting Networks for Image Light Source Manipulation

Contact Info

Product

Resources

About