A Fusion Method of Optical Image and SAR Image Based on Dense-UGAN and Gram–Schmidt Transformation

Kong, Yingying; Hong, Fang; Leung, Henry; Peng, Xiangyang

doi:10.3390/rs13214274

Cited by 27 publications

(13 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Image registration is based on the panchromatic image, and the multispectral image is registered by automatic point selection and manual modification. The Gram-Schmidt (GS) fusion method ( Grochala & Kedzierski, 2017 ; Kong et al, 2021 ; Yilmaz et al, 2020 ), which not only maintains the spectral information of the original image but also retains the spatial texture of the panchromatic band image to the greatest extent, was employed to fuse panchromatic and multispectral image data. The spatial resolution of the multispectral bands was 2.1 m after GS fusion.…”

Section: Methodsmentioning

confidence: 99%

Very high-resolution remote sensing-based mapping of urban residential districts to help combat COVID-19

Guan

Xia

et al. 2022

Cities

View full text Add to dashboard Cite

Section: Methodsmentioning

confidence: 99%

Very high-resolution remote sensing-based mapping of urban residential districts to help combat COVID-19

Guan

Xia

et al. 2022

Cities

View full text Add to dashboard Cite

“…10: Relative performance gain analysis. Approaches dealing with Data Noise: NSCT [95], HRO [97], PIQE [98], ACL-CNN [99], HS2P [9]. Approaches dealing with Label Noise: tRNSL [102], NTDNE [103], AF2GNN [105], RSSC-ETDL [106], CSHLC [107], I-FPFN-EM [108], FFCTL [109], RS-COCL-NLF [110], RVgg19 [111].…”

Section: A Data Noisementioning

confidence: 99%

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

Kieu,

Nguyen,

Nazib

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

In remote sensing (RS), multiple modalities of data are usually available, e.g., RGB, Multispectral, Hyperspectral, LiDAR, and SAR. Multimodal machine learning systems, which fuse these rich multimodal data modalities, have shown better performance compared to unimodal systems. Most multimodal research assumes that all modalities are present, aligned, and noiseless during training and testing time. However, in real-world scenarios, it is common to observe that one or more modalities are missing, noisy, and non-aligned, in either training or testing or both. In addition, acquiring largescale, noise-free annotations is expensive, as a result, lacking sufficient annotated datasets or having to deal with inconsistent labels are open challenges. These challenges can be addressed under a learning paradigm called multimodal co-learning.This paper focuses on multimodal co-learning techniques for remote sensing data. We first review what data modalities are available in the remote sensing domain and the key benefits and challenges of combining multimodal data in the remote sensing context. We then review the remote sensing tasks that would benefit from multimodal processing including classification, segmentation, target detection, anomaly detection, and temporal change detection. We then dive deeper into technical details by reviewing more than 200 recent efforts in this area and provide a comprehensive taxonomy to systematically review state-of-the-art approaches in 4 key co-learning challenges including missing modalities, noisy modalities, limited modality annotations, and weakly-paired modalities. Based on these insights, we propose emerging research directions to inform potential future research in multimodal co-learning for remote sensing.

show abstract

“…In the present section, the proposed work is compared with different fusion methods starting from traditional to advanced deep learning-based algorithms including guided GFF [45], DWT [46], NSCT [47], multi-focus based on gradient image fusion (MWGF) [48], fast filtering image fusion (FFIF) [49], convolutional neural network based on general architecture (IFCNN) [50], and image fusion based on dense blocks (DenseFuse) [51]. Further, the proposed results are compared with two hybrid neural network architectures such as U-GAN + Gram Schmidt transform (U-GAN+GST) [52], and GAN + Non-local means (GAN+NLM) [53] to show the success of the proposed work. Figure 7, Figure 9, Figure 10, and Figure 11 represents the model outcomes of the dataset described in section 3.1.…”

Section: Comparison With Prior Workmentioning

confidence: 99%

PGSOFuse-Net: PCA and GAN Integration for SAROptical Image Fusion

Bandi,

Murugesan,

George

2023

Preprint

View full text Add to dashboard Cite

To illustrate, radar images, popularly known as Synthetic Aperture Radar (SAR) and optical images are very useful in satellite-based information retrieval. But, the complementary nature of both these data led to misinterpretation of the information in various fields like military surveillance, weather forecasting, and resource management. However, the study of image fusion on optical SAR images has overcome some of the limitations in the remote sensing field with cloud removal, change detection, and infrastructure identification. There have been traditional approaches like wavelet analysis, cosine transform, and sparse representation in image fusion which were not more effective in analysing the nature of SAR and optical images. Hence, the advanced algorithms based on neural networks which are very much effective in image fusion, especially in medical diagnostics, social networks, and surveillance are used in the anticipated study. Recent studies of deep-learning-based approaches specifically Generative Adversarial Networks (GAN) architecture suggest decent developments in image fusion. With good developments in Principal Component Analysis for image compression, image segmentation, and image representation in satellite imagery, the suggested work develops a hybrid fusion model which is an innovative combination of Principal Component Analysis and GAN to produce the target image. The effectiveness of the proposed unsupervised algorithm improves the results and preserves the characteristics of both image types. The results specify the importance of the GAN in the SAR optical fusion with the proposed PCA GAN SAR Optical Fusion Network. Subsequently, the assessment of the quality metrics reveals that the current study outperforms prior works and is very useful for future geoinformation.

show abstract

A Fusion Method of Optical Image and SAR Image Based on Dense-UGAN and Gram–Schmidt Transformation

Cited by 27 publications

References 26 publications

Very high-resolution remote sensing-based mapping of urban residential districts to help combat COVID-19

Very high-resolution remote sensing-based mapping of urban residential districts to help combat COVID-19

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

PGSOFuse-Net: PCA and GAN Integration for SAROptical Image Fusion

Contact Info

Product

Resources

About