Multi‐focus image fusion with Siamese self‐attention network

In this paper, a novel conditional focus probability learning model, termed MCNN, is proposed for multi-focus image fusion (MFIF). Given a pair of source images, their conditional focus probabilities can be generated by using the well-trained MCNN, which is further converted into the binary focus masks to directly produce an all-focus image with no postprocessing. To this end, a fully convolutional encoder is designed with two mutually coupled Siamese branches in MCNN, which include a coupling block that bridge between the two branches to provide conditional information to each other, at different layers, such that the encoder can more strongly extract conditional focus features and further encourage the decoder pixel-wisely to give more robust conditional focus probabilities. Moreover, a hybrid loss is designed with a structural sparse fidelity loss and a structural similarity loss to force the network to learn more accurate conditional focus probabilities. Particularly, a convolutional norm with good structural group sparse is proposed, to construct the structural sparse fidelity loss. Simulation results substantiate the superiority of our MCNN over other state-of-the-art, in terms of both visual perception and quantitative evaluation.

Section: Network Backbone For Mfifmentioning

confidence: 99%

“…The typical residual blocks [30] are employed in [31][32][33], whereas the works [34] leverage the dense blocks [35]. Also, various attention mechanisms have been adopted in [36][37][38]. In contrast to the encoders in these methods, the encoders are not always a fully convolutional network.…”

Section: Introductionmentioning

confidence: 99%

MCNN: Conditional focus probability learning to multi‐focus image fusion via mutually coupled neural network

Wang

et al. 2023

“…Li et al [34] proposed a multifocus image fusion based on the deep regression pair learning (DRPL) model, this method converts the entire image into a binary mask for fusion. Guo et al [35] proposed siamese network based on self-attention mechanism, trying to alleviate the local receptive field limitations of convolution operators. However, in these methods, it is still necessary to label images for supervised training of the network.…”

Section: Related Workmentioning

confidence: 99%

“…Guo et al. [35] proposed siamese network based on self‐attention mechanism, trying to alleviate the local receptive field limitations of convolution operators. However, in these methods, it is still necessary to label images for supervised training of the network.…”

Section: Related Workmentioning

confidence: 99%

MCRD‐Net: An unsupervised dense network with multi‐scale convolutional block attention for multi‐focus image fusion

Zhou

Jin

Jiang

et al. 2022

Multi-focus image fusion technology solves the problem of limited depth of field of the optical lens. It can extract different focus parts under the same target to synthesize a fullfocus image. This paper proposes an unsupervised dense network for multi-focus image fusion. In the network, a multi-scale feature extraction module is employed to extract the spatial details of source images from different scales, and a convolutional block attention module is used to select the useful deep features, and a residual module is used to effectively optimize the performance of the network. By introducing these three modules, the proposed network can effectively extract the shallow and deep features of the source images. Besides, Gaussian-based Sum-Modified-Laplacian (GSML) is used to calculate the activity level of the feature map to generate a decision map. The performance of the proposed method is analyzed from two aspects: visual quality and objective metrics. Experimental results show that compared with nine image fusion methods, the performance of this algorithm is better.

“…Recently, convolutional neural networks (CNN) and generative adversarial networks (GAN) based models have been widely concerned for image enhancement tasks [25][26][27]. Due to the excellent ability of feature extraction, CNN-based models have achieved impressive progress in image fusion [28,29], detection [30], and classification [31] tasks. Many CNN-based methods can effectively extract the gradient map and the estimated feature map to reconstruct the enhanced result.…”

Section: Motivation and Preliminarymentioning

confidence: 99%

Adaptive colour restoration and detail retention for image enhancement

Tao

2021

Computer vision-based crowd understanding and analysis technology has been widely used in public safety due to the rapid growth of population and the frequent occurrence of various accidents. Improving imaging quality is the key to improve the performance of crowd analysis, density estimation, target recognition, segmentation, and detection in computer vision tasks. Due to the complex imaging environment such as fog and low illumination, some images taken in outdoor environment often have the problems of colour distortion, lack of details, and the poor imaging quality, which affect the subsequent visual tasks. To improve the imaging quality and visual effect, an adaptive colour restoration and detail retention-based method is proposed for image enhancement. First, to overcome the problem of colour distortion caused by low illumination and fog, a multi-channel fusion based adaptive image colour restoration method is proposed. To make the enhancement result more consistent with human observation, the detail retention-based method is applied to enhance the details. Experimental results demonstrate that the authors' results are effective and outperform the compared methods both in visual and objective evaluations. INTRODUCTIONPublic safety problems have widely attracted attentions due to the frequent occurrence of various safety accidents in the world. Crowd density analysis, detection and intelligent transportation are popular research topics in the field of public safety. Particularly, with the rapid growth of population, crowd counting and analysis have been widely used in video surveillance, traffic control and sports events. In recent years, the development of artificial intelligence technology enables computer vision-based crowd understanding and analysis to develop rapidly. The performance of computer vision-based technology depends on the imaging quality. Therefore, image enhancement is an important technology in computer vision tasks, such as person counting, crowd analysis, re-identification, target segmentation and recognition [1][2][3][4], which is widely used in public safety, military and other visual applications.With the rapid development of deep learning and computer vision technology, many image vision-based intelligent tasks are realized, such as intelligent security and intelligent transportation. Based on the techniques of pattern recognition and machine learning, the person re-identification, vehicle detectionThis is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.