Transformer for Single Image Super-Resolution

Lu, Zhisheng; Li, Juncheng; Liu, Hong; Huang, Chaoyan; Zhang, Linlin; Zeng, Tieyong

doi:10.1109/cvprw56347.2022.00061

Cited by 333 publications

(141 citation statements)

References 24 publications

Supporting

Mentioning

140

Contrasting

Unclassified

Order By: Relevance

“…Further, Nie et al [66] recently proposed depthwise separable convolution to improve the speed up of SR architecture. Recently, the transformers for natural language processing has further prompted researchers to adapt it for computer vision applications, including superresolution [67]- [70] where it is noted promising to improve accuracy. However, even in such frameworks, a SR model is generally trained using a supervised manner with LR images created by bicubic down-sampling.…”

Section: B Supervised Sr Methodsmentioning

confidence: 99%

Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images

et al. 2022

View full text Add to dashboard Cite

Single Image Super-Resolution (SISR) using Convolutional Neural Networks (CNNs) for many applications in supervised manner has resulted in state-of-the-art results. Such supervised models achieve remarkable accuracy; albeit their poor generalization ability for real-world Low-Resolution (LR) images. Supervised training in many SR works involves synthetically generated LR images from its corresponding High-Resolution (HR) images. As the distribution of such LR observation is relatively different from that of real LR image, the supervised training in SISR task results degradation when applied on real-world data. SISR has been scaled to real-world data recently by posing the unsupervised problem into a supervised one through learning the distribution of noisy LR observation first, following which supervised training is performed to obtain the SR image. It therefore involves two steps where the accuracy of SR image relies on how closely the LR's distribution is learnt in the first step. In this work, we overcome such limitation by introducing unsupervised denoising network to transform real noisy LR image to clean image and then pre-trained SR network is utilised to increase the spatial resolution of cleaned LR image to generate SR image. Thus, instead of evaluating the denoised image in LR space to train the denoising network, we inspect the denoised image in SR space which allows to overcome the SR network's generalization problem. The proposed Unsupervised Denoising framework for Super-Resolution (UDSR) is validated on real-world datasets (NTIRE-2020 Real-World SR Challenge validation and testing dataset (Track-1)) by comparing it with many recent unsupervised SISR methods. The performance of denoising and SR networks is superior in terms of various perceptual indices such as Perceptual Index (PI) and Ma Score in addition to numerous non-references metrics.

show abstract

Section: B Supervised Sr Methodsmentioning

confidence: 99%

Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Recently, vision transformers applied to SR have also been proposed, such as the Encoder-Decoder-based Transformer (EDT) [46], Efficient SR Transformer (ESRT) [47], and the Swin Image Restoration (SwinIR) [11] approach that is based on the Swin Transformer [48]. Approaches such as Efficient Long-Range Attention Network (ELAN) [12] and Hybrid Attention Transformer (HAT) [49], which attempt to combine CNN and transformer architectures, have also been proposed with further improvements in SR performance.…”

Section: Non-blind Sr Methodsmentioning

confidence: 99%

The Best of Both Worlds: a Framework for Combining Degradation Prediction with High Performance Super-Resolution Networks

Aquilina¹,

Ciantar²,

Galea³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…NAS and HW-NAS for CNN models have found their way from a simple task on a small dataset to a complex application on a gigantic dataset. However, the Transformer NAS methods have not been specialized for a wide variety of tasks and datasets, even though the manual design of Transformers has been used in diverse tasks such as Super Resolution [244], Semantic Segmentation [245], Medical Image Segmentation [246], etc. Also, the algorithmic breakthrough of search algorithms allowed the use for other purposes, such as Winograd Convolution Search [247], Mixed Precision Quantization Search [123], Fault-Tolerant network search [248], etc.…”

Section: G Applications and Purposesmentioning

confidence: 99%

Neural Architecture Search for Transformers: A Survey

et al. 2022

View full text Add to dashboard Cite

Transformer-based Deep Neural Network architectures have gained tremendous interest due to their effectiveness in various applications across Natural Language Processing (NLP) and Computer Vision (CV) domains. These models are the de facto choice in several language tasks, such as Sentiment Analysis and Text Summarization, replacing Long Short Term Memory (LSTM) model. Vision Transformers (ViTs) have shown better model performance than traditional Convolutional Neural Networks (CNNs) in vision applications while requiring significantly fewer parameters and training time. The design pipeline of a neural architecture for a given task and dataset is extremely challenging as it requires expertise in several interdisciplinary areas such as signal processing, image processing, optimization and allied fields. Neural Architecture Search (NAS) is a promising technique to automate the architectural design process of a Neural Network in a data-driven way using Machine Learning (ML) methods. The search method explores several architectures without requiring significant human effort, and the searched models outperform the manually built networks. In this paper, we review Neural Architecture Search techniques, targeting the Transformer model and its family of architectures such as Bidirectional Encoder Representations from Transformers (BERT) and Vision Transformers. We provide an in-depth literature review of approximately 50 state-of-theart Neural Architecture Search methods and explore future directions in this fast-evolving class of problems.

show abstract

Transformer for Single Image Super-Resolution

Cited by 333 publications

References 24 publications

Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images

Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images

The Best of Both Worlds: a Framework for Combining Degradation Prediction with High Performance Super-Resolution Networks

Neural Architecture Search for Transformers: A Survey

Contact Info

Product

Resources

About