Satellite Image Super-Resolution via Multi-Scale Residual Deep Neural Network

Abstract

Recently, the application of satellite remote sensing images is becoming increasingly popular, but the observed images from satellite sensors are frequently in low-resolution (LR). Thus, they cannot fully meet the requirements of object identification and analysis. To utilize the multi-scale characteristics of objects fully in remote sensing images, this paper presents a multi-scale residual neural network (MRNN). MRNN adopts the multi-scale nature of satellite images to reconstruct high-frequency information accurately for super-resolution (SR) satellite imagery. Different sizes of patches from LR satellite images are initially extracted to fit different scale of objects. Large-, middle-, and small-scale deep residual neural networks are designed to simulate differently sized receptive fields for acquiring relative global, contextual, and local information for prior representation. Then, a fusion network is used to refine different scales of information. MRNN fuses the complementary high-frequency information from differently scaled networks to reconstruct the desired high-resolution satellite object image, which is in line with human visual experience (“look in multi-scale to see better”). Experimental results on the SpaceNet satellite image and NWPU-RESISC45 databases show that the proposed approach outperformed several state-of-the-art SR algorithms in terms of objective and subjective image qualities.

Hyperspectral and Multispectral Remote Sensing Image Fusion Based on Endmember Spatial Information

“…So, a ground truth image does not exist for this dataset. In this case, we use mean gradient (MG) [36] to evaluate the fusion results. MG is defined as follows:…”

Adaptive CNN Ensemble for Complex Multispectral Image Analysis

“…Later, numerous efforts have been made to improvise this method further. For example, a study [18] presents the multiscale residual neural network (MRNN). MRNN adopts the multiscale nature of satellite images to reconstruct high-frequency information accurately for SR.…”

“…In the GCP server (us-west1-b region), we installed the Compute Engine 2//For feature extraction task (2) Perform convolution operation with F � 64 and nonlinearity, via equations 3and 4(3) Add Gaussian noise, via equation 6(4) Apply max-pooling, via equation 7(5) Activate and deactivate neurons using dropout with p, via equation 8 Forward accuracy to the performance feedback module (5) Determine the ensemble accuracies using voting, via Algorithm 2 //Spectral band stream are classifying with their respective instances in the DEC module (6) if % accuracy for B ≥ //if sample does not misclassify (7) Repeat steps 3, 4, and 5 (8) if % accuracy for B ≤ //if sample misclassify (9) Save the B //Save misclassified sample in training repository //as potential new spectral band (10) Counter++ (11) Repeat steps 3, 4, and 5 (12) if counter � 50 //number of misclassified instances reached to 50 (13) Cluster M c using K-mean where K � 1, [35]. //Cluster all the misclassified data samples using the K-means approach with //K � 1, K � 1 the case is assigned to the class of its nearest neighbor (14) Determine optimized centroid, [35]//to optimize the similar sample instances (15) Compare cosine distance cluster sample with cluster centroid, via equation (12) //to segregate most relevant samples in cluster (16) Assign the all nearest samples a hypothetical class X � B n+1 //A new class with additional spectral band information (17) Create new instance classifier i n+1 //New Single Instance, 20 layered architecture (18) Train new instance classifier I � i n+1 with hypothetical class X � B n+1 , via Table 3 //Online training with selected hyperparameters as depicted in Table 3 ALGORITHM 3: Continued.…”

Remote Sensing Imagery Super Resolution Based on Adaptive Multi-Scale Feature Fusion Network

“…However, due to factors such as long-distance imaging, atmospheric turbulence, transmission noise, and motion blurring, the quality and the spatial resolution of remote sensing imagery are relatively poorer and lower as compared with natural images. Moreover, the ground objects of remote sensing imagery usually have different scales, causing the objects and surrounding environment to mutually couple in the joint distribution of their image patterns [2]. Therefore, super-resolution for remote sensing imagery has attracted huge interest and become a hot research topic.…”

“…Dong et al [26] used enhanced residual block and residual channel attention group to obtain multi-level remote sensing feature information. Lu et al [2] reconstructed high-resolution remote sensing images by extracting patches of different sizes as multi-scale information input networks and fusing high-frequency information of different scales. However, the problem of redundant feature information is often ignored.…”