A Real-Time FPGA Implementation of Infrared and Visible Image Fusion Using Guided Filter and Saliency Detection

Zhang, Ling; Yang, Xuefei; Wan, Zhenlong; Cao, Dingxin; Lin, Yingcheng

doi:10.3390/s22218487

Cited by 4 publications

(2 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, infrared imaging technology utilizes differences in infrared radiation intensity for object detection, thereby making it less susceptible to varying lighting and weather conditions [1]. Therefore, infrared image enhancement has become a hot topic in current research and has the potential to bring significant benefits to the field of multi-modal information fusion [2][3][4][5]. Simultaneously, it is extensively employed in fields such as target detection [6][7][8] and medical diagnostics [9,10].…”

Section: Introductionmentioning

confidence: 99%

Multi-Scale FPGA-Based Infrared Image Enhancement by Using RGF and CLAHE

Liu,

Zhou,

Wan

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

Infrared sensors capture thermal radiation emitted by objects. They can operate in all weather conditions and are thus employed in fields such as military surveillance, autonomous driving, and medical diagnostics. However, infrared imagery poses challenges such as low contrast and indistinct textures due to the long wavelength of infrared radiation and susceptibility to interference. In addition, complex enhancement algorithms make real-time processing challenging. To address these problems and improve visual quality, in this paper, we propose a multi-scale FPGA-based method for real-time enhancement of infrared images by using rolling guidance filter (RGF) and contrast-limited adaptive histogram equalization (CLAHE). Specifically, the original image is first decomposed into various scales of detail layers and a base layer using RGF. Secondly, we fuse detail layers of diverse scales, then enhance the detail information by using gain coefficients and employ CLAHE to improve the contrast of the base layer. Thirdly, we fuse the detail layers and base layer to obtain the image with global details of the input image. Finally, the proposed algorithm is implemented on an FPGA using advanced high-level synthesis tools. Comprehensive testing of our proposed method on the AXU15EG board demonstrates its effectiveness in significantly improving image contrast and enhancing detail information. At the same time, real-time enhancement at a speed of 147 FPS is achieved for infrared images with a resolution of 640 × 480.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Scale FPGA-Based Infrared Image Enhancement by Using RGF and CLAHE

Liu,

Zhou,

Wan

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Therefore, extracting the significant features of the fusion image is one of the central problems. Over the past few decades, numerous fusion methods have been proposed by researchers, which can be roughly divided into two categories: traditional fusion methods [ 5 , 6 , 7 ] and deep learning-based fusion methods [ 8 , 9 , 10 ]. Traditional fusion methods measure pixels’ salience in the spatial domain or transform domain, and later design specific fusion rules to fuse them to obtain the fused image.…”

Section: Introductionmentioning

confidence: 99%

DSA-Net: Infrared and Visible Image Fusion via Dual-Stream Asymmetric Network

Yin

Yang

Huang

et al. 2023

Sensors

View full text Add to dashboard Cite

Infrared and visible image fusion technologies are used to characterize the same scene using diverse modalities. However, most existing deep learning-based fusion methods are designed as symmetric networks, which ignore the differences between modal images and lead to source image information loss during feature extraction. In this paper, we propose a new fusion framework for the different characteristics of infrared and visible images. Specifically, we design a dual-stream asymmetric network with two different feature extraction networks to extract infrared and visible feature maps, respectively. The transformer architecture is introduced in the infrared feature extraction branch, which can force the network to focus on the local features of infrared images while still obtaining their contextual information. The visible feature extraction branch uses residual dense blocks to fully extract the rich background and texture detail information of visible images. In this way, it can provide better infrared targets and visible details for the fused image. Experimental results on multiple datasets indicate that DSA-Net outperforms state-of-the-art methods in both qualitative and quantitative evaluations. In addition, we also apply the fusion results to the target detection task, which indirectly demonstrates the fusion performances of our method.

show abstract