Lightweight Attended Multi-Scale Residual Network for Single Image Super-Resolution

Yan, Yitong; Xue, Xinyu; Chen, Wenhui; Peng, Xinyi

doi:10.1109/access.2021.3069775

Cited by 14 publications

(5 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al [66] first incorporated SE [20] with SR and pushed the stateof-the-art performance of SISR. More recent works, such as [9,21,29,43,44,58,59,61], extend this idea by adopting different spatial attention mechanisms or designing advanced attention blocks.…”

Section: Attention Based Sr Methodsmentioning

confidence: 99%

Frequency-Based Enhancement Network for Efficient Super-Resolution

Behjati¹,

Rodríguez

Fernández³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Recently, deep convolutional neural networks (CNNs) have provided outstanding performance in single image super-resolution (SISR). Despite their remarkable performance, the lack of highfrequency information in the recovered images remains a core problem. Moreover, as the networks increase in depth and width, deep CNN-based SR methods are faced with the challenge of computational complexity in practice. A promising and under-explored solution is to adapt the amount of compute based on the different frequency bands of the input. To this end, we present a novel Frequency-based Enhancement Block (FEB) which explicitly enhances the information of high frequencies while forwarding low-frequencies to the output. In particular, this block efficiently decomposes features into low-and high-frequency and assigns more computation to high-frequency ones. Thus, it can help the network generate more discriminative representations by explicitly recovering finer details. Our FEB design is simple and generic and can be used as a direct replacement of commonly used SR blocks with no need to change network architectures. We experimentally show that when replacing SR blocks with FEB we consistently improve the reconstruction error, while reducing the number of parameters in the model. Moreover, we propose a lightweight SR model -Frequency-based Enhancement Network (FENet) -based on FEB that matches the performance of larger models. Extensive experiments demonstrate that our proposal performs favorably against the state-ofthe-art SR algorithms in terms of visual quality, memory footprint, and inference time. The code is available at https://github.com/pbehjatii/FENet.

show abstract

Section: Attention Based Sr Methodsmentioning

confidence: 99%

Frequency-Based Enhancement Network for Efficient Super-Resolution

Behjati¹,

Rodríguez

Fernández³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Compared with CNN, transformer has a better global modeling capability. Inspired by the significant achievements of multi-scale processing in the domains of image restoration [40][41][42] and 3D shape representation, 43 we have introduced the concept of multi-scale into our network. This enhancement allows TMSDNet to further explore the deep multi-scale representation of the object in the voxel feature domain by extracting multi-scale global voxel features from the voxel features output by the combined-transformer block.…”

Section: Overviewmentioning

confidence: 99%

TMSDNet: Transformer with multi‐scale dense network for single and multi‐view 3D reconstruction

Zhu,

Yao,

Zhang

et al. 2023

Computer Animation & Virtual

View full text Add to dashboard Cite

Abstract3D reconstruction is a long‐standing problem. Recently, a number of studies have emerged that utilize transformers for 3D reconstruction, and these approaches have demonstrated strong performance. However, transformer‐based 3D reconstruction methods tend to establish the transformation relationship between the 2D image and the 3D voxel space directly using transformers or rely solely on the powerful feature extraction capabilities of transformers. They ignore the crucial role played by deep multi‐scale representation of the object in the voxel feature domain, which can provide extensive global shape and local detail information about the object in a multi‐scale manner. In this article, we propose a novel framework TMSDNet (transformer with multi‐scale dense network) for single‐view and multi‐view 3D reconstruction with transformer to solve this problem. Based on our well‐designed combined‐transformer Block, which is canonical encoder–decoder architecture, voxel features with spatial order can be extracted from the input image, which are used to further extract multi‐scale global features in parallel using a multi‐scale residual attention module. Furthermore, a residual dense attention block is introduced for deep local features extraction and adaptive fusion. Finally, the reconstructed objects are produced with the voxel reconstruction block. Experiment results on the benchmarks such as ShapeNet and Pix3D datasets demonstrate that TMSDNet outperforms the existing state‐of‐the‐art reconstruction methods substantially.

show abstract

“…On the other hand, so-called lightweight SR models, aiming to reduce the number of parameters and FLOPs while maintaining performance, is actively being studied [8,9,10,11,22,23,37,38,39]. For instance, FSRCNN [22] utilized deconvolution layers to achieve faster processing speed compared to SRCNN.…”

Section: B Lightweight Super-resolution Modelsmentioning

confidence: 99%

TAKDSR: Teacher Assistant Knowledge Distillation Framework for Graphics Image Super-Resolution

Yoon,

Lee,

Song

2023

IEEE Access

View full text Add to dashboard Cite

This paper presents a framework for effectively applying knowledge distillation (KD) to superresolution (SR) tasks for computer graphics (CG) images. Specifically, we propose TAKDSR, a KD framework for SR using a teacher assistant (TA) network. Recently, the performance of SR models has improved dramatically thanks to the development of deep learning. SR models have evolved into a form that requires a considerable amount of computation and parameters while adopting a complex neural network structure to improve performance. However, it is difficult to utilize conventional high-performance SR models for real-time up-scaling in CG applications requiring high resolution and high frame rate. To solve this, we employ an approach that applies KD to a lightweight SR model. At this time, if the high-resolution (HR) image is used as input for the teacher to show superior performance to the student, a large performance difference occurs between the two due to the excessive performance of the teacher. As a result, the teacher's knowledge has a significantly hard and complex nature, and when transferred to the student, the effect of KD can be rather weakened. Therefore, we adopt a TA network to facilitate the propagation of knowledge between teacher and student. At the same time, the distribution of compact features (CF), which are the decoder input of the teacher, is discretized so that it is compatible with the input distribution of the student, enabling effective KD. Experimental results demonstrate the proposed TAKDSR significantly improves the performance of a given SR model on CG image datasets.

show abstract

Lightweight Attended Multi-Scale Residual Network for Single Image Super-Resolution

Cited by 14 publications

References 36 publications

Frequency-Based Enhancement Network for Efficient Super-Resolution

Frequency-Based Enhancement Network for Efficient Super-Resolution

TMSDNet: Transformer with multi‐scale dense network for single and multi‐view 3D reconstruction

TAKDSR: Teacher Assistant Knowledge Distillation Framework for Graphics Image Super-Resolution

Contact Info

Product

Resources

About