Attention-Based Adaptive Selection of Operations for Image Restoration in the Presence of Unknown Combined Distortions

Suganuma, Masanori; Liu, Xing; Okatani, Takayuki

doi:10.1109/cvpr.2019.00925

Cited by 90 publications

(66 citation statements)

References 46 publications

(111 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The attention mechanism in computer vision [27][28][29][30][31] is mainly for the algorithm of learning how to focus on the regions of interest and it plays an increasingly important role in solving many vision tasks. In recent years, most of the work of combining visual attention and deep learning is to form attention parameters by introducing a feature mask.…”

Section: Attention Mechanisms In Vision Tasksmentioning

confidence: 99%

Object Tracking in RGB-T Videos Using Modal-Aware Attention Network and Competitive Learning

Hui

Zhang

et al. 2020

Sensors

106

View full text Add to dashboard Cite

Object tracking in RGB-thermal (RGB-T) videos is increasingly used in many fields due to the all-weather and all-day working capability of the dual-modality imaging system, as well as the rapid development of low-cost and miniaturized infrared camera technology. However, it is still very challenging to effectively fuse dual-modality information to build a robust RGB-T tracker. In this paper, an RGB-T object tracking algorithm based on a modal-aware attention network and competitive learning (MaCNet) is proposed, which includes a feature extraction network, modal-aware attention network, and classification network. The feature extraction network adopts the form of a two-stream network to extract features from each modality image. The modal-aware attention network integrates the original data, establishes an attention model that characterizes the importance of different feature layers, and then guides the feature fusion to enhance the information interaction between modalities. The classification network constructs a modality-egoistic loss function through three parallel binary classifiers acting on the RGB branch, the thermal infrared branch, and the fusion branch, respectively. Guided by the training strategy of competitive learning, the entire network is fine-tuned in the direction of the optimal fusion of the dual modalities. Extensive experiments on several publicly available RGB-T datasets show that our tracker has superior performance compared to other latest RGB-T and RGB tracking approaches.

show abstract

Section: Attention Mechanisms In Vision Tasksmentioning

confidence: 99%

Object Tracking in RGB-T Videos Using Modal-Aware Attention Network and Competitive Learning

Hui

Zhang

et al. 2020

Sensors

106

View full text Add to dashboard Cite

show abstract

“…In order to better restore images of mixed degraded types, the RL-Restore proposed by Yu et al constructs a toolbox that contains small-scale convolutional networks for different restoration tasks, and selects appropriate according to different images to be restored The tool gradually restores damaged images [30]. The OWAN proposed by Suganuma et al performs multiple basic operations in parallel in its core module, and selects the appropriate operation to restore the image according to the specific conditions of the image to be restored [31]. Bai et al proposed an adaptive restoration algorithm based on hierarchical feature fusion.…”

Section: Introductionmentioning

confidence: 99%

Generative Adversarial Unsupervised Image Restoration in Hybrid Degradation Scenes

Tang¹,

Zhu²,

Hu³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we propose an unsupervised blind restoration model for images in hybrid degradation scenes. The proposed model encodes the content information and degradation information of images and then uses the attention module to disentangle the two kinds of information. It can improve the ability of disentangled presentation learning for a generative adversarial network (GAN) to restore the images in hybrid degradation scenes, enhance the detailed features of restored image and remove the artifact combining the adversarial loss, cycle-consistency loss, and perception loss. The experimental results on the DIV2K dataset and medical images show that the proposed method outperforms existing unsupervised image restoration algorithms in terms of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and subjective visual evaluation.

show abstract

“…As reported in [1], the test classification accuracy of ImageNets has been improved substantially compared with other methods at that time. Furthermore, besides classification tasks, attention mechanism has been also used in many other tasks such as object detection [3], [4], semantic segmentation [5], [6], super resolution [7], [8], action recognition [9], [10], etc. As the most popular attention method, SE technology used pooling operators to achieve the invariant feature of each channel, bringing nonlinearity at the same time.…”

Section: Introductionmentioning

confidence: 99%

Kernel Product Neural Networks

Zhou

Shen

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Attention is an important field to explore the importance of each convolutional kernel channel/weight. The existing attention methods mostly use the Squeeze-and-Excitation (SE) technology to extract the global nonlinear feature vectors as the weights of corresponding feature maps. However, the pooling operators and fully-connected layers used in SE technology extract global features at the cost of much valuable information loss and the parameter amount increase. Actually, the feature map containing full information is a ready-made and better attention for other feature maps in the same layer. Simultaneously the products of feature maps will bring powerful non-linearity. Seeing this, Kernel Product (KP) technology is proposed to simply get useful nonlinear attention. To verify the effectiveness of KP, the proposed KP module is employed on Selective Kernel Networks (SKNets) to take the place of the original SE technology. The variety of SKNets is called Kernel Product Networks (KPNets) in this paper. In addition, identity mapping is used to solve the non-convergence problem in very deep neural networks. The KPNets are evaluated on ImageNet-1k, CIFAR-10, and CIFAR-100. The experiment results show that KPNets outperform many state-of-the-art methods and get a similar but more efficient performance than its SKNets with counterpart.

show abstract

Attention-Based Adaptive Selection of Operations for Image Restoration in the Presence of Unknown Combined Distortions

Cited by 90 publications

References 46 publications

Object Tracking in RGB-T Videos Using Modal-Aware Attention Network and Competitive Learning

Object Tracking in RGB-T Videos Using Modal-Aware Attention Network and Competitive Learning

Generative Adversarial Unsupervised Image Restoration in Hybrid Degradation Scenes

Kernel Product Neural Networks

Contact Info

Product

Resources

About