An Infrared and Visible Image Fusion Algorithm Based on LSWT-NSST

Li, Junwu; Li, Binhua; Jiang, Yaoxi

doi:10.1109/access.2020.3028088

Cited by 20 publications

(6 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lifting Wavelet Transform has the advantages of adaptive design, irregular sampling, and integral transform over DWT [41]. Additional techniques include lifting Stationary Wavelet Transform [42], redundant-lifting non-separable Wavelet multi-directional analysis [43], spectral graph Wavelet Transforms [44], quaternion Wavelet Transform, motion-compensated Wavelet Transform, multi-Wavelet, and other fusion methods being applied at the feature level due to their spatial characteristics. Gao et al [45] used the non-subsampled contourlet transform (NSCT) for its flexibility and for being fully shift-invariant.…”

Section: Related Workmentioning

confidence: 99%

Fused Thermal and RGB Imagery for Robust Detection and Classification of Dynamic Objects in Mixed Datasets via Pre-Trained High-Level CNN

Ben-Shoushan

Brook

2023

Remote Sensing

View full text Add to dashboard Cite

Smart vehicles with embedded Autonomous Vehicle (AV) technologies are currently equipped with different types of mounted sensors, aiming to ensure safe movement for both passengers and other road users. The sensors’ ability to capture and gather data to be synchronically interpreted by neural networks for a clear understanding of the surroundings is influenced by lighting conditions, such as natural lighting levels, artificial lighting effects, time of day, and various weather conditions, such as rain, fog, haze, and extreme temperatures. Such changing environmental conditions are also known as complex environments. In addition, the appearance of other road users is varied and relative to the vehicle’s perspective; thus, the identification of features in a complex background is still a challenge. This paper presents a pre-processing method using multi-sensorial RGB and thermal camera data. The aim is to handle issues arising from the combined inputs of multiple sensors, such as data registration and value unification. Foreground refinement, followed by a novel statistical anomaly-based feature extraction prior to image fusion, is presented. The results met the AV challenges in CNN’s classification. The reduction of the collected data and its variation level was achieved. The unified physical value contributed to the robustness of input data, providing a better perception of the surroundings under varied environmental conditions in mixed datasets for day and night images. The method presented uses fused images, robustly enriched with texture and feature depth and reduced dependency on lighting or environmental conditions, as an input for a CNN. The CNN was capable of extracting and classifying dynamic objects as vehicles and pedestrians from the complex background in both daylight and nightlight images.

show abstract

Section: Related Workmentioning

confidence: 99%

Fused Thermal and RGB Imagery for Robust Detection and Classification of Dynamic Objects in Mixed Datasets via Pre-Trained High-Level CNN

Ben-Shoushan

Brook

2023

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Moreover, Tan et al [ 18 ] proposed an image fusion via NSST and PCNN [ 19 ] in the multiscale morphological gradient (MSMG) domain and explored a new fusion method in the MSMG domain. On the basis of NSST, Li et al [ 20 ] proposed the LSWT-NSST image fusion algorithm, which effectively solved edge blurring and Gibbs phenomena in the traditional wavelet transform algorithm and the loss of subtle features in the NSST.…”

Section: Introductionmentioning

confidence: 99%

Infrared and Visible Image Fusion Algorithm Based on Double-Domain Transform Filter and Contrast Transform Feature Extraction

Ma,

Li,

Deng

et al. 2024

Sensors

View full text Add to dashboard Cite

Current challenges in visible and infrared image fusion include color information distortion, texture detail loss, and target edge blur. To address these issues, a fusion algorithm based on double-domain transform filter and nonlinear contrast transform feature extraction (DDCTFuse) is proposed. First, for the problem of incomplete detail extraction that exists in the traditional transform domain image decomposition, an adaptive high-pass filter is proposed to decompose images into high-frequency and low-frequency portions. Second, in order to address the issue of fuzzy fusion target caused by contrast loss during the fusion process, a novel feature extraction algorithm is devised based on a novel nonlinear transform function. Finally, the fusion results are optimized and color-corrected by our proposed spatial-domain logical filter, in order to solve the color loss and edge blur generated in the fusion process. To validate the benefits of the proposed algorithm, nine classical algorithms are compared on the LLVIP, MSRS, INO, and Roadscene datasets. The results of these experiments indicate that the proposed fusion algorithm exhibits distinct targets, provides comprehensive scene information, and offers significant image contrast.

show abstract

“…Processing the source image from different sensors on the same target or scene according to a certain strategy, removing redundant information, and performing multidirectional and multiangle fusion can obtain a more accurate fusion image that reflects the target scene information [2]. With the development of image fusion technology, the cost of hardware equipment is reduced, and more comprehensive and reliable detailed information can be obtained [3]. Therefore, infrared and visible light image fusion technology has been extensively studied by scholars [4].…”

Section: Introductionmentioning

confidence: 99%

“…Infrared sensors can perceive heat radiation of different wavelengths, which can capture hidden target contour information, and they have great night vision and fog penetration capabilities, but they cannot obtain detailed information and have poor resolution [6]. Visible light sensors characterize objects through spectral reflection, and the results have high resolution and rich background information, which are suitable for human visual perception, but the image quality is easily affected by the environment, especially at night and under low visibility conditions [3]. If the images from these two sensors are fused, i.e., the main target information is obtained from the infrared image, while the main detailed background information is obtained from the visible image, then the fused image can provide better target features and more detailed scene information [7].…”

Section: Introductionmentioning

confidence: 99%

“…Visible light sensors characterize objects through spectral reflection, and the results have high resolution and rich background information, which are suitable for human visual perception, but the image quality is easily affected by the environment, especially at night and under low visibility conditions [3]. If the images from these two sensors are fused, i.e., the main target information is obtained from the infrared image, while the main detailed background information is obtained from the visible image, then the fused image can provide better target features and more detailed scene information [7]. Therefore, infrared and visible image fusion technology is a research hotspot, and it plays a key role in target tracking and detection, face recognition and other fields [8], [9].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Efficient Algorithm to Highlight Details in Infrared and Visible Image Fusion

Liu

Zhang

et al. 2021

IEEE Access

View full text Add to dashboard Cite

To improve the fusion quality of infrared and visible images and highlight target and scene details, in this paper, a novel infrared and visible image fusion algorithm is proposed. First, a method for combining dynamic range compression and contrast restoration based on a guided filter is adopted to enhance the contrast of visible source images. Second, guided filter-based image multiscale decomposition is used to decompose images into base layers and detail layers. For base layer fusion, a fusion strategy based on the detail and energy measurements of the source image is proposed to determine the pixel value of the fused image base layer such that the energy loss of the fusion can be reduced and the texture detail features are highlighted to obtain more source image details. Finally, recursive separation and weighted histogram equalization methods are applied to optimize the fused image. Experimental results show that the fusion algorithm and fusion strategy proposed in this paper can effectively improve fusion image clarity, while more detailed target and scene information can still be retained.

show abstract

An Infrared and Visible Image Fusion Algorithm Based on LSWT-NSST

Cited by 20 publications

References 47 publications

Fused Thermal and RGB Imagery for Robust Detection and Classification of Dynamic Objects in Mixed Datasets via Pre-Trained High-Level CNN

Fused Thermal and RGB Imagery for Robust Detection and Classification of Dynamic Objects in Mixed Datasets via Pre-Trained High-Level CNN

Infrared and Visible Image Fusion Algorithm Based on Double-Domain Transform Filter and Contrast Transform Feature Extraction

An Efficient Algorithm to Highlight Details in Infrared and Visible Image Fusion

Contact Info

Product

Resources

About