Advanced Multi-Sensor Optical Remote Sensing for Urban Land Use and Land Cover Classification: Outcome of the 2018 IEEE GRSS Data Fusion Contest

Abstract

This paper presents the scientific outcomes of the 2018 Data Fusion Contest organized by the Image Analysis and Data Fusion Technical Committee of the IEEE Geoscience and Remote Sensing Society. The 2018 Contest addressed the problem of urban observation and monitoring with advanced multi-source optical remote sensing (multispectral LiDAR, hyperspectral imaging, and very high-resolution imagery). The competition was based on urban land use and land cover classification, aiming to distinguish between very diverse and detailed classes of urban objects, materials, and vegetation. Besides data fusion, it also quantified the respective assets of the novel sensors used to collect the data. Participants proposed elaborate approaches rooted in remote-sensing, and also in machine learning and computer vision, to make the most of the available data. Winning approaches combine convolutional neural networks with subtle earth-observation data scientist expertise. Index Terms-Convolutional neural networks (CNN), deep learning, hyperspectral (HS) imaging (HSI), image analysis and data fusion, multimodal, multiresolution, multisource, multispectral light detection and ranging (LiDAR).

Multitask Learning of Height and Semantics From Aerial Images

“…a) Semantic segmentation: this task consists in giving a class label to each pixel in the image [11] and has been commonly carried out in the recent years by Fully-Convolutional Networks (FCNs) since [12]. In remote sensing, it corresponds to the old problem of land-surface classification [13] and has been popularized again by recent benchmarks on urban landuse mapping [9], [8]. Current state-of-the-art approaches based on FCNs include [1], [2] or [14] which combines segmentation with boundary detection.…”

“…Current state-of-the-art approaches based on FCNs include [1], [2] or [14] which combines segmentation with boundary detection. When multi-source data is available, as in the 2018 DFC, dedicated network architectures such as Fusion-CNN [8] can be designed to use this information.…”

Semantic Segmentation of LiDAR Points Clouds: Rasterization Beyond Digital Elevation Models

“…We chose the multi-spectral LiDAR acquisition of the University of Houston issued from 2018 IEEE GRSS Data Fusion Contest dataset [11] to support our experiments. The associated ground truth map has a spatial resolution of 0.5 m. The original 20 classes have been reduced to 7 generic urban classes (roads, grass, trees, residential buildings, nonresidential buildings, cars and trains) to evaluate the overall accuracy.…”

A Point-Wise LiDAR and Image Multimodal Fusion Network (PMNet) for Aerial Point Cloud 3D Semantic Segmentation

“…IEEE-GRSS Data Fusion is well known in remote sensing society and well organized benchmark dataset for multi-modal fusion challenges [30][31][32][33]. The University of Houston campus dataset has provided by the National Center for Airborne Laser Mapping (NCALM).…”