Fusion of SAR and optical images using pixel-based CNN

Abstract

Sensors of different wavelengths in remote sensing field capture data. Each and every sensor has its own capabilities and limitations. Synthetic aperture radar (SAR) collects data that has a high spatial and radiometric resolution. The optical remote sensors capture images with good spectral information. Fused images from these sensors will have high information when implemented with a better algorithm resulting in the proper collection of data to predict weather forecasting, soil exploration, and crop classification. This work encompasses a fusion of optical and radar data of Sentinel series satellites using a deep learning-based convolutional neural network (CNN). The three-fold work of the image fusion approach is performed in CNN as layered architecture covering the image transform in the convolutional layer, followed by the activity level measurement in the max pooling layer. Finally, the decision-making is performed in the fully connected layer. The objective of the work is to show that the proposed deep learning-based CNN fusion approach overcomes some of the difficulties in the traditional image fusion approaches. To show the performance of the CNN-based image fusion, a good number of image quality assessment metrics are analyzed. The consequences demonstrate that the integration of spatial and spectral information is numerically evident in the output image and has high robustness. Finally, the objective assessment results outperform the state-of-the-art fusion methodologies.

Research on denoising and enhancing methods of medical images based on convolutional neural networks

“…MSE and PSNR only include pixel-by-pixel variations, while SSIM also takes into account perceptual factors, which makes it a more realistic representation of the way viewers perceive image quality. 18 The subjective absolute score refers to people's score on the image itself, while the subjective relative score refers to people's score on a group of similar images. The scoring standard of the subjective absolute score of the image is shown in Table 1, and the subjective relative score of the image is shown in Table 2.…”

“…Compared to mean squared error (MSE) and peak signal‐to‐noise ratio (PSNR), structural similarity index (SSIM) more nearly approximates human visual perception since it takes into account brightness, contrast, and structural information. MSE and PSNR only include pixel‐by‐pixel variations, while SSIM also takes into account perceptual factors, which makes it a more realistic representation of the way viewers perceive image quality 18 . The subjective absolute score refers to people's score on the image itself, while the subjective relative score refers to people's score on a group of similar images.…”