Geospatial Computer Vision Based on Multi-Modal Data—How Valuable Is Shape Information for the Extraction of Semantic Information?

Abstract

Abstract:In this paper, we investigate the value of different modalities and their combination for the analysis of geospatial data of low spatial resolution. For this purpose, we present a framework that allows for the enrichment of geospatial data with additional semantics based on given color information, hyperspectral information, and shape information. While the different types of information are used to define a variety of features, classification based on these features is performed using a random forest classifier. To draw conclusions about the relevance of different modalities and their combination for scene analysis, we present and discuss results which have been achieved with our framework on the MUUFL Gulfport Hyperspectral and LiDAR Airborne Data Set.

Application of Machine and Deep Learning Strategies for the Classification of Heritage Point Clouds

“…In recent years, significant progress has come out in automatic procedures for classification of point clouds or meshes thanks to the advent of Machine Learning approaches (Hackel et al, 2016;Weinmann et al, 2017;Wang et al, 2018). Several benchmarks have been proposed in the Geomatics community, providing labelled terrestrial and airborne data on which users can test and validate their algorithms.…”

Semantic Segmentation of Aerial Imagery via Multi-Scale Shuffling Convolutional Neural Networks With Deep Supervision

“…Few investigations involve very basic handcrafted features given by the Normalized Difference Vegetation Index (NDVI) and the normalized Digital Surface Model (nDSM) (Gerke, 2014;Audebert et al, 2016;Liu et al, 2017). Other kinds of hand-crafted radiometric or geometric features which can be extracted from a local image neighborhood (Gerke and Xiao, 2014;Weinmann and Weinmann, 2018) have however only rarely been involved, although, in the context of classifying aerial imagery based on given true orthophotos and the corresponding DSMs, it has recently been demonstrated that the additional consideration of such hand-crafted radiometric and geometric features on a per-pixel basis may lead to improved classification results (Chen et al, 2018b). In this paper, we focus on a multi-scale extension of Shuffling Convolutional Neural Networks (Chen et al, 2018a;Chen et al, 2018b) involving deep supervision, and we thereby also involve a diversity of hand-crafted radiometric and geometric features extracted from the true orthophotos and their corresponding DSMs, respectively.…”

“…Furthermore, we consider geometric features in the form of local 3D shape features extracted from the DSM. Based on the spatial 3D coordinates corresponding to a local 3 × 3 image neighborhood, we efficiently derive the 3D structure tensor (Weinmann and Weinmann, 2018) and normalize its three eigenvalues by their sum. The normalized eigenvalues, in turn, are then used to calculate the features of linearity (L), planarity (P), sphericity (S), omnivariance (O), anisotropy (A), eigenentropy (E) and change of curvature (E) (West et al, 2004;Pauly et al, 2003) which have been involved in a variety of investigations for 3D scene analysis (Demantké et al, 2011;Weinmann, 2016;Hackel et al, 2016).…”

“…Given data in the form of true orthophotos and the corresponding DSMs, the semantic segmentation of aerial imagery has been addressed by extracting hand-crafted features and using standard classifiers such as Random Forests (Weinmann and Weinmann, 2018;Gerke and Xiao, 2014) or Conditional Random Fields (CRFs) (Gerke, 2014). In recent years, however, the use of modern deep learning techniques has become increasingly popular, as such techniques are famous for their representation capacity and their ability of learning features.…”

From 2d to 3d Supervised Segmentation and Classification for Cultural Heritage Applications

“…Many algorithms require a fine-tuning of different parameters depending upon the nature of data and applications. The majority of these are supervised methods, where a training phase is mandatory and fundamental to guide the successive machine learning classification solution (Guo et al, 2014;Niemeyer et al, 2014;Xu et al, 2014;Weinmann et al, 2015;Hackel et al, 2016;Qi et al, 2016;Weinmann et al, 2017;Wang et al, 2018).…”