UIFGAN: An unsupervised continual-learning generative adversarial network for unified image fusion

Le, Zhuliang; Huang, Jun; Xu, Han; Fan, Fan; Ma, Yong; Mei, Xiaoguang; Ma, Jiayi

doi:10.1016/j.inffus.2022.07.013

Cited by 35 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this network, a full-size jump-connected generator is applied to extract shallow features, and the discriminator uses two Markov discriminators to fully retain the valid information in the infrared and visible images by playing adversarial games with the generator. In addition, a novel intensity masking generative adversarial network (IM GAN) [ 26 ] and an unsupervised continual-learning generative adversarial network (UIFGAN) [ 27 ] were designed to complement multimodal image information, whereas, it fails to integrate the extracted features efficiently. Xu et al [ 12 ] introduced attention mechanisms to the fusion network for feature extraction, while Liu [ 28 ] proposed an Attention-guided and Wavelet-constrained Generative Adversarial Network for infrared and visible image fusion(AWFGAN) model based on Generative Adversarial Nets (GAN), which could better preserve important information of the raw images.…”

Section: Relevant Workmentioning

confidence: 99%

FDNet: An end-to-end fusion decomposition network for infrared and visible images

Di,

Ren,

Liu

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

Infrared and visible image fusion can generate a fusion image with clear texture and prominent goals under extreme conditions. This capability is important for all-day climate detection and other tasks. However, most existing fusion methods for extracting features from infrared and visible images are based on convolutional neural networks (CNNs). These methods often fail to make full use of the salient objects and texture features in the raw image, leading to problems such as insufficient texture details and low contrast in the fused images. To this end, we propose an unsupervised end-to-end Fusion Decomposition Network (FDNet) for infrared and visible image fusion. Firstly, we construct a fusion network that extracts gradient and intensity information from raw images, using multi-scale layers, depthwise separable convolution, and improved convolution block attention module (I-CBAM). Secondly, as the FDNet network is based on the gradient and intensity information of the image for feature extraction, gradient and intensity loss are designed accordingly. Intensity loss adopts the improved Frobenius norm to adjust the weighing values between the fused image and the two raw to select more effective information. The gradient loss introduces an adaptive weight block that determines the optimized objective based on the richness of texture information at the pixel scale, ultimately guiding the fused image to generate more abundant texture information. Finally, we design a single and dual channel convolutional layer decomposition network, which keeps the decomposed image as possible with the input raw image, forcing the fused image to contain richer detail information. Compared with various other representative image fusion methods, our proposed method not only has good subjective vision, but also achieves advanced fusion performance in objective evaluation.

show abstract

Section: Relevant Workmentioning

confidence: 99%

FDNet: An end-to-end fusion decomposition network for infrared and visible images

Di,

Ren,

Liu

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…In terms Neck networks, the target detection network adds an FPN structure between the Backbone and the last Head output layer. The Head output layer has the same anchor frame mechanism as the YOLOv4 [15], the main improvement is about loss function during training, GIOU_Loss, and the DIOU_nms of the prediction box filtering.…”

Section: Yolov5 Detectionmentioning

confidence: 99%

The Effect of Data Augmentation Methods on Pedestrian Object Detection

Liu

Wei

2022

Electronics

View full text Add to dashboard Cite

Night landscapes are a key area of monitoring and security as information in pictures caught on camera is not comprehensive. Data augmentation gives these limited datasets the most value. Considering night driving and dangerous events, it is important to achieve the better detection of people at night. This paper studies the impact of different data augmentation methods on target detection. For the image data collected at night under limited conditions, three different types of enhancement methods are used to verify whether they can promote pedestrian detection. This paper mainly explores supervised and unsupervised data augmentation methods with certain improvements, including multi-sample augmentation, unsupervised Generative Adversarial Network (GAN) augmentation and single-sample augmentation. It is concluded that the dataset obtained by the heterogeneous multi-sample augmentation method can optimize the target detection model, which can allow the mean average precision (mAP) of a night image to reach 0.76, and the improved Residual Convolutional GAN network, the unsupervised training model, can generate new samples with the same style, thus greatly expanding the dataset, so that the mean average precision reaches 0.854, and the single-sample enhancement of the deillumination can greatly improve the image clarity, helping improve the precision value by 0.116.

show abstract

“…Reference [ [18] , [19] , [20] ] captured the multilevel features of the source images via residual learning. Moreover, modern GAN-based approaches [ [21] , [22] , [23] , [24] , [25] , [26] , [27] , [28] , [29] , [30] ] exploit multi-granularity convolution kernels of the same feature level, yielding different receptive fields and in turn improving fusion performance. For example, each network layer of the feature extractor in Refs.…”

Section: Introductionmentioning

confidence: 99%

“…For example, each network layer of the feature extractor in Refs. [ [21] , [22] , [23] , [24] ] utilized convolution kernels of different sizes to extract useful information from source images. Li [ 25 , 26 ] introduced multi-grained attention network to enable the fusion model to perceive the target region or detail information of the source image from multiple scales.…”

Section: Introductionmentioning

confidence: 99%

Image fusion using Y-net-based extractor and global-local discriminator

Yang,

Zhu,

Wang

et al. 2024

Heliyon

View full text Add to dashboard Cite

UIFGAN: An unsupervised continual-learning generative adversarial network for unified image fusion

Cited by 35 publications

References 42 publications

FDNet: An end-to-end fusion decomposition network for infrared and visible images

FDNet: An end-to-end fusion decomposition network for infrared and visible images

The Effect of Data Augmentation Methods on Pedestrian Object Detection

Image fusion using Y-net-based extractor and global-local discriminator

Contact Info

Product

Resources

About