Fusion of Heterogeneous Earth Observation Data for the Classification of Local Climate Zones

Ghamisi, Pedram; Zhu, Xiao Xiang

doi:10.1109/tgrs.2019.2914967

Cited by 26 publications

(11 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another promising approach is supervised automatic LCZ classification with satellite remote sensing images (Yokoya et al, 2017, Qiu et al, 2018, Zhang et al, in press, Hu et al, 2018), which can potentially be used for LCZ mapping on a worldwide scale without relying on local expert knowledge for each individual city (Mills et al, 2015). However, large-scale LCZ mapping remains difficult because of the unavailability of sufficient high-quality training data, which hinders the generalization of trained classifiers.…”

Section: Introductionmentioning

confidence: 99%

Local climate zone-based urban land cover classification from multi-seasonal Sentinel-2 images with a recurrent residual network

Qiu

Mou

Schmitt

et al. 2019

ISPRS Journal of Photogrammetry and Remote Sensing

Self Cite

135

View full text Add to dashboard Cite

The local climate zone (LCZ) scheme was originally proposed to provide an interdisciplinary taxonomy for urban heat island (UHI) studies. In recent years, the scheme has also become a starting point for the development of higher-level products, as the LCZ classes can help provide a generalized understanding of urban structures and land uses. LCZ mapping can therefore theoretically aid in fostering a better understanding of spatio-temporal dynamics of cities on a global scale. However, reliable LCZ maps are not yet available globally. As a first step toward automatic LCZ mapping, this work focuses on LCZ-derived land cover classification, using multi-seasonal Sentinel-2 images. We propose a recurrent residual network (Re-ResNet) architecture that is capable of learning a joint spectral-spatial-temporal feature representation within a unitized framework. To this end, a residual convolutional neural network (ResNet) and a recurrent neural network (RNN) are combined into one end-to-end architecture. The ResNet is able to learn rich spectral-spatial feature representations from single-seasonal imagery, while the RNN can effectively analyze temporal dependencies of multi-seasonal imagery. Cross validations were carried out on a diverse dataset covering seven distinct European cities, and a quantitative analysis of the experimental results revealed that the combined use of the multi-temporal information and Re-ResNet results in an improvement of approximately 7 percent points in overall accuracy. The proposed framework has the potential to produce consistent-quality urban land cover and LCZ maps on a large scale, to support scientific progress in fields such as urban geography and urban climatology.

show abstract

Section: Introductionmentioning

confidence: 99%

Local climate zone-based urban land cover classification from multi-seasonal Sentinel-2 images with a recurrent residual network

Qiu

Mou

Schmitt

et al. 2019

ISPRS Journal of Photogrammetry and Remote Sensing

Self Cite

135

View full text Add to dashboard Cite

show abstract

“…Finally, a Multilayer Perceptron (MLP) is used to perform per-pixel classification. In [93], the authors perform local climate zones classifications by fusing multitemporal and multispectral satellite images that have different spatial resolutions. The authors propose a method for weighted voting of the output of the classifiers trained with the different data modalities and show more accurate results than feature stacking.…”

Section: Multimodal Techniquesmentioning

confidence: 99%

OpenStreetMap: Challenges and Opportunities in Machine Learning and Remote Sensing

Muñoz

Srivastava

Tuia

et al. 2021

IEEE Geosci. Remote Sens. Mag.

106

View full text Add to dashboard Cite

OpenStreetMap (OSM) is a community-based, freely available, editable map service that was created as an alternative to authoritative ones. Given that it is edited mainly by volunteers with different mapping skills, the completeness and quality of its annotations are heterogeneous across different geographical locations. Despite that, OSM has been widely used in several applications in Geosciences, Earth Observation and environmental sciences. In this work, we present a review of recent methods based on machine learning to improve and use OSM data. Such methods aim either 1) at improving the coverage and quality of OSM layers, typically using GIS and remote sensing technologies, or 2) at using the existing OSM layers to train models based on image data to serve applications like navigation or land use classification. We believe that OSM (as well as other sources of open land maps) can change the way we interpret remote sensing data and that the synergy with machine learning can scale participatory map making and its quality to the level needed to serve global and up-to-date land mapping. A preliminary version of this manuscript has been presented in the first authors dissertation [1].

show abstract

“…To our knowledge, this is the first study conducted to improve LCZ classification accuracy based on building information (area and height) and satellite data through a CNN-based classification approach. While some studies have used building data, such as OSM, as an input variable of CNN or canonical correlation forest (CCF) models [17,45], they did not use the quantitative characteristics of buildings,…”

Section: Impact Of Missing Building Informationmentioning

confidence: 99%

Improving Local Climate Zone Classification Using Incomplete Building Data and Sentinel 2 Images Based on Convolutional Neural Networks

et al. 2020

View full text Add to dashboard Cite

Recent studies have enhanced the mapping performance of the local climate zone (LCZ), a standard framework for evaluating urban form and function for urban heat island research, through remote sensing (RS) images and deep learning classifiers such as convolutional neural networks (CNNs). The accuracy in the urban-type LCZ (LCZ1-10), however, remains relatively low because RS data cannot provide vertical or horizontal building components in detail. Geographic information system (GIS)-based building datasets can be used as primary sources in LCZ classification, but there is a limit to using them as input data for CNN due to their incompleteness. This study proposes novel methods to classify LCZ using Sentinel 2 images and incomplete building data based on a CNN classifier. We designed three schemes (S1, S2, and a scheme fusion; SF) for mapping 50 m LCZs in two megacities: Berlin and Seoul. S1 used only RS images, and S2 used RS and building components such as area and height (or the number of stories). SF combined two schemes (S1 and S2) based on three conditions, mainly focusing on the confidence level of the CNN classifier. When compared to S1, the overall accuracies for all LCZ classes (OA) and the urban-type LCZ (OAurb) of SF increased by about 4% and 7–9%, respectively, for the two study areas. This study shows that SF can compensate for the imperfections in the building data, which causes misclassifications in S2. The suggested approach can be excellent guidance to produce a high accuracy LCZ map for cities where building databases can be obtained, even if they are incomplete.

show abstract

Fusion of Heterogeneous Earth Observation Data for the Classification of Local Climate Zones

Cited by 26 publications

References 39 publications

Local climate zone-based urban land cover classification from multi-seasonal Sentinel-2 images with a recurrent residual network

Local climate zone-based urban land cover classification from multi-seasonal Sentinel-2 images with a recurrent residual network

OpenStreetMap: Challenges and Opportunities in Machine Learning and Remote Sensing

Improving Local Climate Zone Classification Using Incomplete Building Data and Sentinel 2 Images Based on Convolutional Neural Networks

Contact Info

Product

Resources

About