Multi-Sensor Image Fusion: A Survey of the State of the Art

Li, Bing; Yu, Xiaoquan; Zhang, Daqiao; Su, Juan; Hu, Xiaoxiang; Guo, Weilin

doi:10.4236/jcc.2021.96005

Cited by 16 publications

(11 citation statements)

References 162 publications

(136 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…86 However, the approaches based on sparse-based fusion experienced issues with synchronization of sparsity level to loss of relevant information in the medical image recovery process and the problem of quality of the learned dictionary. [87][88][89] In this paper, a modified sparsity-based image fusion framework was proposed. First, 16 natural grayscale images were used to train the dictionary of dimension 64 × 128 with a five per atom sparsity level.…”

Section: Details Of Experiments Done On 38 Brain Tumor Patientsmentioning

confidence: 99%

Medical image fusion by sparse-based modified fusion framework using block total least-square update dictionary learning algorithm

Saini

Mathur

2022

J. Med. Imag.

View full text Add to dashboard Cite

Purpose:The technique of fusing or integrating medical images collected from single and various modalities is known as medical image fusion. This is done to improve the quality of the images and combine information from several medical images. The whole procedure aids medical practitioners in gaining correct information from single images. Image fusion is one of the fastest-growing research topics in the medical imaging field. Sparse modeling is a popular signal representation technique used for image fusion with different dictionary learning approaches. We propose a medical image fusion with sparse representation (SR) and block total least-square (BLOTLESS) update dictionary learning.Approach: The domain of dictionary learning is the most significant research domain related to SR. An efficient dictionary increases the effectiveness of sparse modeling. Due to SR being an ongoing interesting research area, the medical image fusion process is done with a modified image fusion framework with recently developed BLOTLESS update dictionary learning. Results:The experimental results are compared for the image fusion process using other stateof-the-art dictionary learning algorithms, such as simultaneous codeword optimization, method of optimal directions, and K-singular value decomposition. The effectiveness of the algorithm is evaluated based on image fusion quantitative parameters. Results show that the BLOTLESS update dictionary algorithm is a promising modification for the sparse-based image fusion with its applicability in the fusion of images related to different diseases. Conclusions:The experiments and results show that the dictionary learning algorithm plays an important role in the sparse-based image fusion general framework. The fusion results also show that the proposed improved image fusion framework for medical images is promising compared with frameworks with other dictionary learning algorithms. As an application, it is also used as a tool for the fusion of different modularities of images related to brain tumor and glioma.

show abstract

Section: Details Of Experiments Done On 38 Brain Tumor Patientsmentioning

confidence: 99%

Medical image fusion by sparse-based modified fusion framework using block total least-square update dictionary learning algorithm

Saini

Mathur

2022

J. Med. Imag.

View full text Add to dashboard Cite

show abstract

“…Li B et al, to solve the problem that the mean-shift algorithm cannot update the target model in realtime, improved the mean-shift algorithm and proposed the Camshift algorithm, which is a tracking algorithm based on color probability distribution. is algorithm is suitable for tracking scenarios where the target color is single and has a large color difference with the background, but not for target tracking scenarios where the target and background colors are similar or the background is complex and the target texture is rich [19]. Luo L et al proposed a continuous adaptive mean drift tracking algorithm with a background suppression histogram model, which improves the tracking accuracy and stability by suppressing the hues belonging to the background in the original color model [20].…”

Section: Panoramic Video Multitarget Real-time Trackingmentioning

confidence: 99%

Real-Time Multitarget Tracking for Panoramic Video Based on Dual Neural Networks for Multisensor Information Fusion

Lin

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

A multitarget real-time tracking system for panoramic video with multisensor information fusion dual neural networks is studied and implemented by combining dual neural networks, fused geometric features, and deep learning-based target real-time tracking algorithms. The motion model of multisensor information fusion is analyzed, the dual neural network perturbation model is introduced, and the state variables of the system are determined. Combined with the data structure model of multisensor information fusion, a least-squares-based optimization function is constructed. In this optimization function, the image sensor is constructed as an observationally constrained residual term based on the pixel plane, the inertial measurement unit is constructed as an observationally constrained residual term based on the motion model, and the motion relationship between sensors is constructed as a motion-constrained residual term to derive a mathematical model for real-time multitarget tracking of panoramic video. The simulation experimental results show that the integrated position estimation accuracy of the multisensor information fusion dual neural network-based panoramic video multitarget real-time tracking fusion algorithm is 10.5% higher than that of the optimal distributed panoramic video multitarget fusion algorithm without feedback. The results of the simulation experiments show that the integrated position estimation accuracy of the multitarget real-time tracking fusion algorithm for panoramic video improves by 10.96%, and the integrated speed estimation accuracy improves by 6.32% compared with the optimal distributed multitarget fusion algorithm without feedback. The proposed panoramic video multitarget real-time tracking algorithm based on the dual neural network can effectively improve the target tracking accuracy of the model on degraded frames (motion blur, target occlusion, out-of-focus, etc.), and the stability of the algorithm for target location and category detection is effectively improved by multiframe feature fusion. The research in this paper provides better technical support and research theory for panoramic video multitarget real-time tracking.

show abstract

“…2) Feature-level fusion: Feature-level fusion first extracts the relevant features of each source data, such as shape, size, texture, etc., then feature fusion processing is performed on the features to generate new features, and finally object interpretation work is performed. At present, the existing feature fusion techniques can be divided into two main categories [29]: feature selection-based and feature extractionbased. In feature selection-based fusion techniques, all the features are first assembled together, and then feature selection is performed using a suitable method and finally used for object interpretation.…”

Section: Introductionmentioning

confidence: 99%

“…Decision-level fusion can use a variety of logical inference methods, statistical methods, information theory methods, etc., mainly including Bayesian inference, Dempster-Shafer (D-S) evidence theory [33], fuzzy decision making, neural networks, etc. [29]. Rastiveis [34] proposed a three-level fusion algorithm based on Bayesian theory for LiDAR data and aerial imagery to improve urban object coverage classification accuracy.…”

Section: Introductionmentioning

confidence: 99%

Multiview Learning for Impervious Surface Mapping Using High-Resolution Multispectral Imagery and LiDAR Data

Luo

Yang

Feng

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The use of multi-source remote sensing data to obtain urban impervious surface has become a popular research topic. Multi-source remote sensing data fusion techniques can provide object interpretation with a higher accuracy. However, most decision-level fusion methods make insufficient use of the complementary information and degree of association between similar object data. To fill this gap, in this paper, we propose a dual-view learning fusion classification method (DvLF) based on multi-view learning. First, DvLF uses co-training algorithm to combine multiple data sources for accurate classification, extracting easy-to-classify area while separating difficult-toclassify regions for further analysis. Secondly, a canonical correlation analysis method is adopted to mine the degree of association of similar object data for constructing a subspace projection field of each object sample. The data in the difficult-toclassify regions are classified in the projection field of each object, and then the results of each classification are fused by voting. Finally, the classification results of the two regions are combined into the classification results of the whole image to achieve impervious surface mapping. The proposed method is applied to the dual-sensor (high-resolution image and LiDAR) Buffalo dataset and the dual-sensor (RGB and multispectral LiDAR) Houston dataset. The experimental results show that our method achieved a significant improvement in classification accuracy compared to other methods. The overall classification accuracy of this new DvLF fusion method on the Buffalo and Houston datasets is 83.35% and 88.84%, respectively leading to accurate highresolution impervious surface mapping.

show abstract

Multi-Sensor Image Fusion: A Survey of the State of the Art

Cited by 16 publications

References 162 publications

Medical image fusion by sparse-based modified fusion framework using block total least-square update dictionary learning algorithm

Medical image fusion by sparse-based modified fusion framework using block total least-square update dictionary learning algorithm

Real-Time Multitarget Tracking for Panoramic Video Based on Dual Neural Networks for Multisensor Information Fusion

Multiview Learning for Impervious Surface Mapping Using High-Resolution Multispectral Imagery and LiDAR Data

Contact Info

Product

Resources

About