Multiscale channel attention network for infrared and visible image fusion

Zhu, Jiahui; Dou, Qingyu; Jian, Lihua; Liu, Kai; Hussain, Fawad; Yang, Xiaobo

doi:10.1002/cpe.6155

Cited by 9 publications

(5 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We considered reducing the weight of the low-resolution soft tissues of the CT image and increase the weight of dense structures. This idea was similar to MCAFusion [ 57 ]. MCAFusion employed VSM [ 34 ] to extract the visual salient features of infrared and visible images and then fused those features.…”

Section: Discussionmentioning

confidence: 99%

Multi-Scale Mixed Attention Network for CT and MRI Image Fusion

Liu

Yan

Zhang

et al. 2022

Entropy

Self Cite

View full text Add to dashboard Cite

Recently, the rapid development of the Internet of Things has contributed to the generation of telemedicine. However, online diagnoses by doctors require the analyses of multiple multi-modal medical images, which are inconvenient and inefficient. Multi-modal medical image fusion is proposed to solve this problem. Due to its outstanding feature extraction and representation capabilities, convolutional neural networks (CNNs) have been widely used in medical image fusion. However, most existing CNN-based medical image fusion methods calculate their weight maps by a simple weighted average strategy, which weakens the quality of fused images due to the effect of inessential information. In this paper, we propose a CNN-based CT and MRI image fusion method (MMAN), which adopts a visual saliency-based strategy to preserve more useful information. Firstly, a multi-scale mixed attention block is designed to extract features. This block can gather more helpful information and refine the extracted features both in the channel and spatial levels. Then, a visual saliency-based fusion strategy is used to fuse the feature maps. Finally, the fused image can be obtained via reconstruction blocks. The experimental results of our method preserve more textual details, clearer edge information and higher contrast when compared to other state-of-the-art methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Multi-Scale Mixed Attention Network for CT and MRI Image Fusion

Liu

Yan

Zhang

et al. 2022

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…A densely connected network has two channel inputs is used as a feature extraction module and a deeper recombination module is designed to increase the retention of information [14]. On this basis, a deep dense residual network is proposed, which can increase the amount of fused image information or embed attention mechanism modules in the network, selectively providing more brightness and gradient information for fused image [15][16] . In response to the weak generalization ability of the deep learning fusion model mentioned above, the adaptive information retention degree is proposed [17] , which automatically evaluates and obtains the importance parameters of the original image.…”

Section: Fusion Methods Based On Convolutional Neural Networkmentioning

confidence: 99%

Review of deep learning fusion methods for visible and Infrared thermal images

Zhang,

Yun,

Jiang

et al. 2024

International Conference on Frontiers of Applied Optics and Computer Engineering (AOCE 2024)

View full text Add to dashboard Cite

Image fusion technology can integrate visible and infrared thermal image from different sensors and retain the feature data and complementary information in fused images. Condition monitoring of transmission lines is beneficial to improve the stability for power grid operation. Image sensors that are developed rapidly in transmission line application has advantages such as non-contact measuring, automatic monitoring and unmanned aerial vehicle carrying. With the development of machine learning technology, many deep learning networks are used in visible and infrared thermal image fusion of transmission line. Firstly, the image fusion methods based on convolutional neural network, autoencoder network and generative adversarial network which advantages and applicable conditions of different frameworks are compared. Secondly, the evaluation indicators of fused images are proposed, and the combined evaluation method based on multiple indicators are illustrated. Then, focused with the demand of transmission line operation condition monitoring, different deep learning algorithm frameworks have been verification by visible and infrared thermal images gathered from the transmission line. Finally, these algorithms are compared with evaluation indicators and the research direction of the future developments are proposed.

show abstract

“…For example, Munir and Azam et al [ 29 ] considered that multispectral images can provide complementary information for thermal infrared images and proposed an attention-guided feature fusion method to achieve accurate detection of pedestrians in thermal infrared images. Zhu and Dou et al [ 30 ] proposed a multiscale channel attention image fusion method to achieve the fusion of visible light and thermal infrared images while improving the precision of target detection. Considering that the poor resolution of thermal infrared images limits the feature extraction capabilities of the network, Zhang and Xu et al [ 31 ] proposed a new backbone network, Deep-IRobject, to achieve target feature extraction and fusion.…”

Section: Related Workmentioning

confidence: 99%

Non-GDANets: Sports small object detection of thermal images with Non-Glodal decoupled Attention

Zhao¹,

Mao²,

Meng³

et al. 2022

PLoS ONE

View full text Add to dashboard Cite

Because thermal infrared sport targets have rich and complex semantic information, there is a high coupling between different types of features. In view of these limitations, we propose a Non-Glodal decoupled Attention, namely,local U-shaped attention decoupling network (LUANets), which aims to decompose the coupling relationship of different sport target features in thermal infrared images and establish effective spatial dependence between them. This method takes the captured multi-scale initial features according to different levels and inputs them into the local decoupling module with U-shaped attention structure to realize the decomposition of semantic details. At the same time, considering the correlation between different targets, in the process of feature decomposition, using prior knowledge as guiding information many times to establish effective spatial dependence. Secondly, we design a two-way cross-aggregation FPN module to cross-aggregate information flows in the front and back directions to achieve feature interaction while further reducing the coupling between different types of features. The evaluation results on data such as TIIs,SportFCs and FLIR show that the LUANets method we proposed has achieved the best detection performance, with mAP of 68.72%,59.51% and 65.29%, respectively.

show abstract

Multiscale channel attention network for infrared and visible image fusion

Cited by 9 publications

References 56 publications

Multi-Scale Mixed Attention Network for CT and MRI Image Fusion

Multi-Scale Mixed Attention Network for CT and MRI Image Fusion

Review of deep learning fusion methods for visible and Infrared thermal images

Non-GDANets: Sports small object detection of thermal images with Non-Glodal decoupled Attention

Contact Info

Product

Resources

About