Single Image Super-Resolution via a Holistic Attention Network

Niu, Ben; Wen, Weilei; Ren, Wenqi; Zhang, Xiangde; Yang, Lianping; Wang, Shuzhen; Zhang, Kaihao; Cao, Xiaochun; Shen, Haifeng

doi:10.1007/978-3-030-58610-2_12

Cited by 458 publications

(308 citation statements)

References 33 publications

Supporting

Mentioning

262

Contrasting

Order By: Relevance

“…Many existing methods [ 11 , 12 , 16 , 35 , 36 ] carried out multiple complex convolution operations at each downsampling layer to prevent the network from losing important detailed information, which will overload the network with large parameters. In our work, we only carry out a small number of feature extraction operations and then fuse multi-scale hierarchical information in small-scale space.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Attention Mechanism. Inspired by its successful applications in natural language processing, the attention mechanism has been widely used in image processing tasks [19,25,29,[32][33][34][35]. Zhang et al [32] leveraged the attention mechanism to allow the network to focus on the relationship among spatial image areas.…”

Section: Related Workmentioning

confidence: 99%

“…MSPFN [19] combined pyramid structure [31] and channel attention mechanism [30] to synergistically represent multi-scale rain-pattern information. More recently, Niu et al [35] proposed a holistic attention network (HAN) to characterize the relationship between network layers. Nevertheless, HAN regarded multiple feature channels of each layer as a whole group and only focused on estimating the correlation between feature channel groups of different layers.…”

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

A Lightweight Fusion Distillation Network for Image Deblurring and Deraining

Zhang

Liu

et al. 2021

Sensors

View full text Add to dashboard Cite

Recently, deep learning-based image deblurring and deraining have been well developed. However, most of these methods fail to distill the useful features. What is more, exploiting the detailed image features in a deep learning framework always requires a mass of parameters, which inevitably makes the network suffer from a high computational burden. We propose a lightweight fusion distillation network (LFDN) for image deblurring and deraining to solve the above problems. The proposed LFDN is designed as an encoder–decoder architecture. In the encoding stage, the image feature is reduced to various small-scale spaces for multi-scale information extraction and fusion without much information loss. Then, a feature distillation normalization block is designed at the beginning of the decoding stage, which enables the network to distill and screen valuable channel information of feature maps continuously. Besides, an information fusion strategy between distillation modules and feature channels is also carried out by the attention mechanism. By fusing different information in the proposed approach, our network can achieve state-of-the-art image deblurring and deraining results with a smaller number of parameters and outperform the existing methods in model complexity.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

A Lightweight Fusion Distillation Network for Image Deblurring and Deraining

Zhang

Liu

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Back and forth, a blurred image can get the explosive number of frames, which means the model has the potential to interpolate subsequent frames with high accuracy. By exploiting the information embedded in motion blur [37], our model can be employed to disassemble a motion-blurred image into multiple frames, which can break through the limitations of the device to get an intelligent highframe-rate video in the future. It also can be used in many applications, such as video editing and temporal super-resolution of videos.…”

Section: Motion Interpolationmentioning

confidence: 99%

Contiguous Loss for Motion-Based, Non-Aligned Image Deblurring

2021

View full text Add to dashboard Cite

In general dynamic scenes, blurring is the result of the motion of multiple objects, camera shaking or scene depth variations. As an inverse process, deblurring extracts a sharp video sequence from the information contained in one single blurry image—it is itself an ill-posed computer vision problem. To reconstruct these sharp frames, traditional methods aim to build several convolutional neural networks (CNN) to generate different frames, resulting in expensive computation. To vanquish this problem, an innovative framework which can generate several sharp frames based on one CNN model is proposed. The motion-based image is put into our framework and the spatio-temporal information is encoded via several convolutional and pooling layers, and the output of our model is several sharp frames. Moreover, a blurry image does not have one-to-one correspondence with any sharp video sequence, since different video sequences can create similar blurry images, so neither the traditional pixel2pixel nor perceptual loss is suitable for focusing on non-aligned data. To alleviate this problem and model the blurring process, a novel contiguous blurry loss function is proposed which focuses on measuring the loss of non-aligned data. Experimental results show that the proposed model combined with the contiguous blurry loss can generate sharp video sequences efficiently and perform better than state-of-the-art methods.

show abstract

Organic, Inorganic, Hybrid, and Bio Materials Research at Sungkyunkwan University

Lee

2019

Advanced Materials

View full text Add to dashboard Cite

Recent deep-learning-based single image super-resolution (SISR) methods have shown impressive performance whereas typical methods train their networks by minimizing the pixelwise distance with respect to a given high-resolution (HR) image. However, despite the basic training scheme being the predominant choice, its use in the context of ill-posed inverse problems has not been thoroughly investigated. In this work, we aim to provide a better comprehension of the underlying constituent by decomposing target HR images into two subcomponents: (1) the optimal centroid which is the expectation over multiple potential HR images, and (2) the inherent noise defined as the residual between the HR image and the centroid. Our findings show that the current training scheme cannot capture the ill-posed nature of SISR and becomes vulnerable to the inherent noise term, especially during early training steps. To tackle this issue, we propose a novel optimization method that can effectively remove the inherent noise term in the early steps of vanilla training by estimating the optimal centroid and directly optimizing toward the estimation. Experimental results show that the proposed method can effectively enhance the stability of vanilla training, leading to overall performance gain. Codes are available at github.com/2minkyulee/ECO.

show abstract

Single Image Super-Resolution via a Holistic Attention Network

Cited by 458 publications

References 33 publications

A Lightweight Fusion Distillation Network for Image Deblurring and Deraining

A Lightweight Fusion Distillation Network for Image Deblurring and Deraining

Contiguous Loss for Motion-Based, Non-Aligned Image Deblurring

Organic, Inorganic, Hybrid, and Bio Materials Research at Sungkyunkwan University

Contact Info

Product

Resources

About