Multispectral Image Fusion and Colorization

Blasch, Erik; Zheng, Yefeng; Liu, Zheng

doi:10.1117/3.2316455

Cited by 40 publications

(13 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When it comes to modalities that involve radio frequency (RF) and electro-optical (EO) sensors, the focus for fusion research has traditionally been on active RF [ 18 ] sensors and [ 19 ] applications. Whether it is the fusion of Synthetic Aperture Radar (SAR) and multi-spectral images [ 20 ] via random forest classifier or double-weighted decision level neural network fusion schemes [ 21 ], many EO/RF sensor fusion applications normally use active RF in one form or another.…”

Section: Literature Reviewmentioning

confidence: 99%

Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data

Vakil

Blasch

Ewing

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

In the Information Age, the widespread usage of blackbox algorithms makes it difficult to understand how data is used. The practice of sensor fusion to achieve results is widespread, as there are many tools to further improve the robustness and performance of a model. In this study, we demonstrate the utilization of a Long Short-Term Memory (LSTM-CCA) model for the fusion of Passive RF (P-RF) and Electro-Optical (EO) data in order to gain insights into how P-RF data are utilized. The P-RF data are constructed from the in-phase and quadrature component (I/Q) data processed via histograms, and are combined with enhanced EO data via dense optical flow (DOF). The preprocessed data are then used as training data with an LSTM-CCA model in order to achieve object detection and tracking. In order to determine the impact of the different data inputs, a greedy algorithm (explainX.ai) is implemented to determine the weight and impact of the canonical variates provided to the fusion model on a scenario-by-scenario basis. This research introduces an explainable LSTM-CCA framework for P-RF and EO sensor fusion, providing novel insights into the sensor fusion process that can assist in the detection and differentiation of targets and help decision-makers to determine the weights for each input.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data

Vakil

Blasch

Ewing

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The registration of the images includes detecting features, setting and comparing them, estimating the transformation models, and converting and re-sampling the image. The process includes fusion rules which are applied either as part of the image transformation models or as direct mathematical applications, such as choosing or averaging the maximum pixel value [10].…”

Section: A Image Fusion Reviewmentioning

confidence: 99%

Combining Multiple Seismic Attributes using Convolutional Neural Networks

Alotaibi¹,

Fadel²,

Jamal³

et al. 2022

IJACSA

View full text Add to dashboard Cite

Seismic exploration involves estimating the properties of the Earth's subsurface from reflected seismic waves then visualizing the resulting seismic data and its attributes. These data and derived seismic attributes provide complementary information and reduce the amount of time and effort for the geoscientist. Multiple conventional methods to combine various seismic attributes exist, but the number of attributes is always limited, and the quality of the resulting image varies. This paper proposes a method that can be used to overcome these limitations. In this paper, we propose using Deep Learning-based image fusion models to combine seismic attributes. By using convolutional neural network (CNN) capabilities in feature extraction, the resulting image quality is better than that obtained with conventional methods. This work implemented two models and conducted a number of experiments using them. Several techniques have been used to evaluate the results, such as visual inspection, and using image fusion metrics. The experiments show that the Image-fusion Framework, using the Image Fusion Framework Based on CNN (IFCNN) approach, outperformed all other models in both quantitative and visual analysis. Its QAB/F and MS-SSIM scores are 50% and 10%, respectively, higher than all other models. Also, IFCNN was evaluated against the current state-of-the-art solution, Octree, in a comparative study. IFCNN overcomes the limitation of the Octree method and succeeds in combining nine seismic attributes with a better-combining quality, with QAB/F and NAB/F scores being 40% higher.

show abstract

“…Image registration entails feature detection, aligning and matching, estimation of the transformation model, image transformation and resampling. Fusion rules are performed either as a direct mathematical application such as averaging or choosing the maximum pixel value or as a part of the image transformation model [9]. DL-based image fusion has shown a lot of potential in improving image fusion techniques.…”

Section: A Overview Of Image Fusionmentioning

confidence: 99%

Enhancement of 3D Seismic Images using Image Fusion Techniques

Alotaibi¹,

Fadel²,

Jamal³

et al. 2021

IJACSA

View full text Add to dashboard Cite

Seismic images are data collected by sending seismic waves to the earth subsurface, recording the reflection and providing subsurface structural information. Seismic attributes are quantities derived from seismic data and provide complementary information. Enhancing seismic images by fusing them with seismic attributes will improve the subsurface visualization and reduce the processing time. In seismic data interpretation, fusion techniques have been used to enhance the resolution and reduce the noise of a single seismic attribute. In this paper, we investigate the enhancement of 3D seismic images using image fusion techniques and neural networks to combine seismic attributes. The paper evaluates the feasibility of using image fusion models pretrained on specific image fusion tasks. These models achieved the best results on their respective tasks and are tested for seismic image fusion. The experiments showed that image fusion techniques are capable of combining up to three seismic attributes without distortion, future studies can increase the number. This is the first study conducted using pretrained models on other types of images for seismic image fusion and the results are promising.

show abstract

Multispectral Image Fusion and Colorization

Cited by 40 publications

References 2 publications

Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data

Finding Explanations in AI Fusion of Electro-Optical/Passive Radio-Frequency Data

Combining Multiple Seismic Attributes using Convolutional Neural Networks

Enhancement of 3D Seismic Images using Image Fusion Techniques

Contact Info

Product

Resources

About