FusionCNN: a remote sensing image fusion algorithm based on deep convolutional neural networks

Ye, Fajie; Li, Xiongfei; Zhang, Xiaoli

doi:10.1007/s11042-018-6850-3

Cited by 37 publications

(18 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…CNNs were previously used in remote sensing for land use or land cover classification using hyperspectral [26], multispectral data [27], cloud detection [28], building detection [29], and image fusion [30]. Works employing CNNs usually enjoy high accuracies that are often higher than the previous state-of-the-art algorithms.…”

Section: Introductionmentioning

confidence: 99%

Asbestos—Cement Roofing Identification Using Remote Sensing and Convolutional Neural Networks (CNNs)

et al. 2020

View full text Add to dashboard Cite

Due to the pathogenic nature of asbestos, a statutory ban on asbestos-containing products has been in place in Poland since 1997. In order to protect human health and the environment, it is crucial to estimate the quantity of asbestos–cement products in use. It has been evaluated that about 90% of them are roof coverings. Different methods are used to estimate the amount of asbestos–cement products, such as the use of indicators, field inventory, remote sensing data, and multi- and hyperspectral images; the latter are used for relatively small areas. Other methods are sought for the reliable estimation of the quantity of asbestos-containing products, as well as their spatial distribution. The objective of this paper is to present the use of convolutional neural networks for the identification of asbestos–cement roofing on aerial photographs in natural color (RGB) and color infrared (CIR) compositions. The study was conducted for the Chęciny commune. Aerial photographs, each with the spatial resolution of 25 cm in RGB and CIR compositions, were used, and field studies were conducted to verify data and to develop a database for Convolutional Neural Networks (CNNs) training. Network training was carried out using the TensorFlow and R-Keras libraries in the R programming environment. The classification was carried out using a convolutional neural network consisting of two convolutional blocks, a spatial dropout layer, and two blocks of fully connected perceptrons. Asbestos–cement roofing products were classified with the producer’s accuracy of 89% and overall accuracy of 87% and 89%, depending on the image composition used. Attempts have been made at the identification of asbestos–cement roofing. They focus primarily on the use of hyperspectral data and multispectral imagery. The following classification algorithms were usually employed: Spectral Angle Mapper, Support Vector Machine, object classification, Spectral Feature Fitting, and decision trees. Previous studies undertaken by other researchers showed that low spectral resolution only allowed for a rough classification of roofing materials. The use of one coherent method would allow data comparison between regions. Determining the amount of asbestos–cement products in use is important for assessing environmental exposure to asbestos fibres, determining patterns of disease, and ultimately modelling potential solutions to counteract threats.

show abstract

Section: Introductionmentioning

confidence: 99%

Asbestos—Cement Roofing Identification Using Remote Sensing and Convolutional Neural Networks (CNNs)

et al. 2020

View full text Add to dashboard Cite

show abstract

“…A sparse representation image fusion algorithm called Laplacian pyramid sparse representation (LPSR) [ 44 ] (download from: ) is also in our contrast algorithm. There are also popular deep learning image fusion algorithms, such as Fusion convolutional neural network based algorithm (FusionCNN) [ 45 ] and dual-discriminator conditional generative adversarial network based algorithm (DDcGAN) [ 46 ]; besides, guided filtering fusion algorithm (GFF) [ 47 ] (download from: ) and internal generative mechanism (IGM) [ 48 ] are also indispensable two contrast algorithms. The code of all the contrast algorithms comes from the relevant papers and some from the relevant academic forums.…”

Section: Resultsmentioning

confidence: 99%

Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space

Guo

Zang

et al. 2020

Entropy

Self Cite

View full text Add to dashboard Cite

In order to obtain the physiological information and key features of source images to the maximum extent, improve the visual effect and clarity of the fused image, and reduce the computation, a multi-modal medical image fusion framework based on feature reuse is proposed. The framework consists of intuitive fuzzy processing (IFP), capture image details network (CIDN), fusion, and decoding. First, the membership function of the image is redefined to remove redundant features and obtain the image with complete features. Then, inspired by DenseNet, we proposed a new encoder to capture all the medical information features in the source image. In the fusion layer, we calculate the weight of each feature graph in the required fusion coefficient according to the trajectory of the feature graph. Finally, the filtered medical information is spliced and decoded to reproduce the required fusion image. In the encoding and image reconstruction networks, the mixed loss function of cross entropy and structural similarity is adopted to greatly reduce the information loss in image fusion. To assess performance, we conducted three sets of experiments on medical images of different grayscales and colors. Experimental results show that the proposed algorithm has advantages not only in detail and structure recognition but also in visual features and time complexity compared with other algorithms.

show abstract

“…Due to its powerful learning ability, deep learning has been extensively researched in image processing, including RSIF [19][20][21] [35]. In 2018, Shao et al [19] proposed an RSIF method that introduced a double-branch network structure based on the deep convolutional neural network.…”

Section: A Remote Sensing Image Fusion Based On Deep Learningmentioning

confidence: 99%

“…In 2019, Liu et al [21] proposed a two-stream fusion network to extract the features of PAN and MS images for fusion and producing a final image. In the same year, Ye et al [35] proposed an image fusion algorithm based on the convolutional neural network and using a fusion model with end-to-end attributes, in which the input was a pair of source images and the output was a fused image. Compared with conventional RSIF methods, deep learning-based methods can extract and fuse features without following any artificial fusion rules.…”

Section: A Remote Sensing Image Fusion Based On Deep Learningmentioning

confidence: 99%

Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure

Jin

Huang

Jiang

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Remote sensing image fusion (RSIF) can generate an integrated image with high spatial and spectral resolution. The fused remote sensing image is conducive to applications including disaster monitoring, ecological environment investigation, and dynamic monitoring. However, most existing deep learning-based RSIF methods require ground truths (or reference images) to train a model, and the acquisition of ground truths is a difficult problem. To address this, we propose a semi-supervised RSIF method based on the multi-scale conditional generative adversarial networks (cGANs) by combining the multi-skip connection and pseudo-Siamese structure. This new method can simultaneously extract the features of panchromatic and multi-spectral images to fuse them without a ground truth; the adopted multi-skip connection contributes to presenting image details. In addition, we propose a composite loss function, which combines the least squares loss, L1 loss, and peak signal-noise ratio loss to train the model; the composite loss function can help to retain the spatial details and spectral information of the source images. Moreover, we verify the proposed method by extensive experiments, and the results show that the new method can achieve outstanding performance without relying on the ground truth. Index Terms-conditional generative adversarial network; deep learning; image fusion; loss function; remote sensing image fusion I. INTRODUCTION MAGE fusion aims to fuse the complementary information in two or more source images obtained by different sensors so that a new comprehensive image can be generated [1][2][3]. The fused image can present more useful information than any one of the source images. Therefore, image fusion techniques can provide a high-quality fused image to help interpret a scene or target, and related research areas include medical images [5], multi-focus images [3][6], infrared and visible images [7][8], and remote sensing images [9][10][10].Remote sensing images with high spatial and spectral resolution play an important role in geological exploration,

show abstract

FusionCNN: a remote sensing image fusion algorithm based on deep convolutional neural networks

Cited by 37 publications

References 29 publications

Asbestos—Cement Roofing Identification Using Remote Sensing and Convolutional Neural Networks (CNNs)

Asbestos—Cement Roofing Identification Using Remote Sensing and Convolutional Neural Networks (CNNs)

Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space

Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure

Contact Info

Product

Resources

About