Boosting target-level infrared and visible image fusion with regional information coordination

Han, Mina; Yu, Kailong; Qiu, Junhui; Li, Hao; Wu, Dan; Rao, Yujing; Yang, Yang; Xing, Lin; Bai, Haicheng; Zhou, Chengjiang

doi:10.1016/j.inffus.2022.12.005

Cited by 22 publications

(6 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AT‐GAN 2 proposes a generative adversarial network (GAN) with intensity attention modules and semantic transition modules to explore key information in infrared and visible modals. Han et al 1 propose to achieve targeted level image with a scene texture attention module. Moreover, some research has been down to facilitate high‐level vision tasks like object detection and semantic segmentation.…”

Section: Related Workmentioning

confidence: 99%

“…Detecting objects in fused images of visible and infrared modals is a significant task for many applications like traffic surveillance and military reconnaissance. However, in this area, most of research focus on proposing better image fusion methods, 1,2 or designing better infrared object detection. 3,4 In other words, the significance of object detection on fusion images is underrated.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Toward reliable fusion object detection based on dilated pyramid and semantic attention

Chang

Gao

et al. 2023

Engineering Reports

Self Cite

View full text Add to dashboard Cite

Object detection on fused images of visible and infrared modals is of great importance for many applications, for example, surveillance and rescue at low‐light conditions. However, current detectors have difficulty for robust fused image detection for mainly two reasons. First, objects are presented in various shapes and sizes, making some hard samples cannot be localized accurately. Second, the same object category in the fused images will have different appearance due to changing weather condition, temperature and intrinsic heat. Such a contradiction will degrade the classification task of a detection network, since it cannot merge commonalities and distinguish differences well. In this paper, we propose to reconstruct the detection pipeline of current detectors, and enhance the detection ability on difficult samples in fused images. Specifically, a Dilation Pyramid Network (DPN) is designed at the lateral connection to generate and aggregate features of various receptive field, without increasing pyramid layers. To strengthen the classification, a Semantic Category Attention Module (SCAM) is proposed to capture attention centers of semantics in fused images, rather than object centers. Abundant experiments on two fusion datasets show that the proposed method achieves a satisfying performance, and both modules can greatly improve current generic detectors on fused images.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Toward reliable fusion object detection based on dilated pyramid and semantic attention

Chang

Gao

et al. 2023

Engineering Reports

Self Cite

View full text Add to dashboard Cite

show abstract

“…Conversely, models trained on generic datasets do not perform well on content from a particular dataset. For instance, the proposed evaluation methods, based on human visual perceptual properties, have not been effective in evaluating color fusion image databases, 6 – 8 which is largely due to the uniqueness of dual-band color fusion images themselves 9 . Furthermore, many deep learning evaluation algorithms 10 – 14 have proven to be quite effective in the field of no-reference image quality.…”

Section: Introductionmentioning

confidence: 99%

“…For instance, the proposed evaluation methods, based on human visual perceptual properties, have not been effective in evaluating color fusion image databases, [6][7][8] which is largely due to the uniqueness of dual-band color fusion images themselves. 9 Furthermore, many deep learning evaluation algorithms [10][11][12][13][14] have proven to be quite effective in the field of no-reference image quality. However, it is difficult to achieve desirable results for color harmony assessment of dual-band color fused images.…”

Section: Introductionmentioning

confidence: 99%

Dual-band color fusion image quality assessment based on color descriptors

Gao,

Song,

Zhang

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

Significant progress has been made in no-reference image quality assessment in terms of color harmony in recent years. However, most methods do not consider color contrast and lose some color information in the quantification process, so they perform poorly when evaluating the color harmony of color images. An image quality assessment model based on color descriptors is established under the inspiration of natural scene statistics. The model introduces local color contrast features of color fusion images and combines them with visual attention mechanism. The local color statistics coefficients of images in both RGB color space and CIELAB color space are calculated to reduce the loss of color information. Furthermore, a multi-scale content-aware network is used to extract multi-scale content features and establish the connection between color features and image content. Finally, the image quality score is obtained by the quality assessment network. The dataset validation and experimental results reveal that the proposed model, compared with the existing 15 methods, has good agreement with human visual perception and better performance.

show abstract

“…With the development of deep learning, weakly supervised deep learning networks have been used more frequently in image fusion methods. These methods can be roughly summarized as convolutional-neural-network-based fusion methods [11][12][13][14][15] and generative-adversarial-network-based fusion methods [1,2,[16][17][18][19].…”

Section: Introductionmentioning

confidence: 99%

Two-Branch Feature Interaction Fusion Method Based on Generative Adversarial Network

et al. 2023

Self Cite

View full text Add to dashboard Cite

This study proposes a fusion method of infrared and visible images based on feature interaction. Existing fusion methods can be classified into two categories based on a single-branch network and a two-branch network. Generative adversarial networks are widely used in single-branch-based fusion methods, which ignore the difference in feature extraction caused by different input images. Most two-branch-based fusion methods use convolutional neural networks, which do not take into account the inverse promotion of fusion results and lack the interaction between different input features. To remedy the shortcomings of these fusion methods and better utilize the feature from source images, this study proposes a two-branch feature interactions method based on a generative adversarial network for visible and infrared image fusion. In the generator part, a two-branch feature interaction approach was designed to extract features from different inputs and realize feature interaction through the network connection of different branches. In the discriminator part, a double-classification discriminator was used for visible images and infrared images. Extensive comparison experiments with state-of-the-art methods have demonstrated the advantages of this proposed generative adversarial network based on two-branch feature interaction, which can enhance the texture details of objects in fusion results and reduce the interference of noise information from source inputs. In addition, the above advantages were also confirmed in generalization experiments of object detection.

show abstract

Boosting target-level infrared and visible image fusion with regional information coordination

Cited by 22 publications

References 38 publications

Toward reliable fusion object detection based on dilated pyramid and semantic attention

Toward reliable fusion object detection based on dilated pyramid and semantic attention

Dual-band color fusion image quality assessment based on color descriptors

Two-Branch Feature Interaction Fusion Method Based on Generative Adversarial Network

Contact Info

Product

Resources

About