An Object-Based Strategy for Improving the Accuracy of Spatiotemporal Satellite Imagery Fusion for Vegetation-Mapping Applications

Guan, H. R.; Su, Yanjun; Hu, Tianyu; Chen, Jin

doi:10.3390/rs11242927

Cited by 10 publications

(3 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fu et al [32] improved the searching strategy of pixels by accounting for spectral similarity and land cover distribution in the moving window. Guan et al [33] introduced objectoriented constraints to guide the similar pixel selection. Liu et al [34] extracted the phenological information to decrease the uncertainties resulted from similar pixel selection procedure.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning-Based Spatiotemporal Data Fusion Using a Patch-to-Pixel Mapping Strategy and Model Comparisons

Sun

Pan

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

Tradeoffs among the spatial, spectral, and temporal resolutions of satellite sensors make it difficult to acquire remote sensing images at both high spatial and high temporal resolutions from an individual sensor. Studies have developed methods to fuse spatiotemporal data from different satellite sensors, and these methods often assume linear changes in surface reflectance across time and adopt empirical rules and handcrafted features. Here we propose a dense spatiotemporal fusion (DenseSTF) network based on the convolutional neural network to deal with these problems. DenseSTF uses a patch-to-pixel modeling strategy which can provide abundant texture details for each pixel in the target fine image to handle heterogeneous landscapes, and models both forward and backward temporal dependence to account for land cover changes. Moreover, DenseSTF adopts a mapping function with little assumptions and empirical rules which allows for establishing reliable relationships between the coarse and fine images. We tested DenseSTF in three contrast scenes with different degrees of heterogeneity and temporal changes, and made comparisons with three rule-based fusion approaches and three convolutional neural networks. Experimental results indicate that DenseSTF can provide accurate fusion results and outperform the other tested methods, especially when the land cover changes abruptly. The structure of the deep learning networks largely impacts the success of data fusion. Our study developed a novel approach based on the convolutional neural network using a patch-to-pixel mapping strategy and highlighted the effectiveness of the deep learning networks in spatiotemporal fusion of the remote sensing data.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep Learning-Based Spatiotemporal Data Fusion Using a Patch-to-Pixel Mapping Strategy and Model Comparisons

Sun

Pan

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Remote sensing data have been extensively employed for vegetation mapping in various environments due to its ability to discriminate broad scales of land cover types. For vegetation monitoring in urban and rural regions, aerial photography and satellite imaging have been used [ 3 , 4 , 5 , 6 ]. Urban vegetation cover is much more fragmented than natural vegetation (i.e., forest, rangeland), making accurate extraction of vegetation cover more complex and challenging.…”

Section: Introductionmentioning

confidence: 99%

Urban Vegetation Mapping from Aerial Imagery Using Explainable AI (XAI)

Abdollahi

Pradhan

2021

Sensors

View full text Add to dashboard Cite

Urban vegetation mapping is critical in many applications, i.e., preserving biodiversity, maintaining ecological balance, and minimizing the urban heat island effect. It is still challenging to extract accurate vegetation covers from aerial imagery using traditional classification approaches, because urban vegetation categories have complex spatial structures and similar spectral properties. Deep neural networks (DNNs) have shown a significant improvement in remote sensing image classification outcomes during the last few years. These methods are promising in this domain, yet unreliable for various reasons, such as the use of irrelevant descriptor features in the building of the models and lack of quality in the labeled image. Explainable AI (XAI) can help us gain insight into these limits and, as a result, adjust the training dataset and model as needed. Thus, in this work, we explain how an explanation model called Shapley additive explanations (SHAP) can be utilized for interpreting the output of the DNN model that is designed for classifying vegetation covers. We want to not only produce high-quality vegetation maps, but also rank the input parameters and select appropriate features for classification. Therefore, we test our method on vegetation mapping from aerial imagery based on spectral and textural features. Texture features can help overcome the limitations of poor spectral resolution in aerial imagery for vegetation mapping. The model was capable of obtaining an overall accuracy (OA) of 94.44% for vegetation cover mapping. The conclusions derived from SHAP plots demonstrate the high contribution of features, such as Hue, Brightness, GLCM_Dissimilarity, GLCM_Homogeneity, and GLCM_Mean to the output of the proposed model for vegetation mapping. Therefore, the study indicates that existing vegetation mapping strategies based only on spectral characteristics are insufficient to appropriately classify vegetation covers.

show abstract

“…Nowadays, the increasing availability of remote sensing (RS) data offers widespread opportunities in many important application fields, such as urban planning [1][2][3], aerial scene retrieval [4][5][6], change detection [7,8], analysis of the earth's surface [9,10], vegetation mapping [11,12], and remote object detection [13,14]. In these (and many other) important applications, the visual interpretation of RS scenes becomes a particularly challenging task, since a semantic characterization of RS images is required to deal with highly complex spatio-spectral land cover components that lead to high intra-class (and low inter-class) variability [15].…”

Section: Introductionmentioning

confidence: 99%

High-Rankness Regularized Semi-Supervised Deep Metric Learning for Remote Sensing Imagery

Kang

Fernández-Beltrán

et al. 2020

Remote Sensing

View full text Add to dashboard Cite

Deep metric learning has recently received special attention in the field of remote sensing (RS) scene characterization, owing to its prominent capabilities for modeling distances among RS images based on their semantic information. Most of the existing deep metric learning methods exploit pairwise and triplet losses to learn the feature embeddings with the preservation of semantic-similarity, which requires the construction of image pairs and triplets based on the supervised information (e.g., class labels). However, generating such semantic annotations becomes a completely unaffordable task in large-scale RS archives, which may eventually constrain the availability of sufficient training data for this kind of models. To address this issue, we reformulate the deep metric learning scheme in a semi-supervised manner to effectively characterize RS scenes. Specifically, we aim at learning metric spaces by utilizing the supervised information from a small number of labeled RS images and exploring the potential decision boundaries for massive sets of unlabeled aerial scenes. In order to reach this goal, a joint loss function, composed of a normalized softmax loss with margin and a high-rankness regularization term, is proposed, as well as its corresponding optimization algorithm. The conducted experiments (including different state-of-the-art methods and two benchmark RS archives) validate the effectiveness of the proposed approach for RS image classification, clustering and retrieval tasks. The codes of this paper are publicly available.

show abstract

An Object-Based Strategy for Improving the Accuracy of Spatiotemporal Satellite Imagery Fusion for Vegetation-Mapping Applications

Cited by 10 publications

References 53 publications

Deep Learning-Based Spatiotemporal Data Fusion Using a Patch-to-Pixel Mapping Strategy and Model Comparisons

Deep Learning-Based Spatiotemporal Data Fusion Using a Patch-to-Pixel Mapping Strategy and Model Comparisons

Urban Vegetation Mapping from Aerial Imagery Using Explainable AI (XAI)

High-Rankness Regularized Semi-Supervised Deep Metric Learning for Remote Sensing Imagery

Contact Info

Product

Resources

About