Land Cover Classification from fused DSM and UAV Images Using Convolutional Neural Networks

Abstract

In recent years, remote sensing researchers have investigated the use of different modalities (or combinations of modalities) for classification tasks. Such modalities can be extracted via a diverse range of sensors and images. Currently, there are no (or only a few) studies that have been done to increase the land cover classification accuracy via unmanned aerial vehicle (UAV)-digital surface model (DSM) fused datasets. Therefore, this study looks at improving the accuracy of these datasets by exploiting convolutional neural networks (CNNs). In this work, we focus on the fusion of DSM and UAV images for land use/land cover mapping via classification into seven classes: bare land, buildings, dense vegetation/trees, grassland, paved roads, shadows, and water bodies. Specifically, we investigated the effectiveness of the two datasets with the aim of inspecting whether the fused DSM yields remarkable outcomes for land cover classification. The datasets were: (i) only orthomosaic image data (Red, Green and Blue channel data), and (ii) a fusion of the orthomosaic image and DSM data, where the final classification was performed using a CNN. CNN, as a classification method, is promising due to hierarchical learning structure, regulating and weight sharing with respect to training data, generalization, optimization and parameters reduction, automatic feature extraction and robust discrimination ability with high performance. The experimental results show that a CNN trained on the fused dataset obtains better results with Kappa index of~0.98, an average accuracy of 0.97 and final overall accuracy of 0.98. Comparing accuracies between the CNN with DSM result and the CNN without DSM result for the overall accuracy, average accuracy and Kappa index revealed an improvement of 1.2%, 1.8% and 1.5%, respectively. Accordingly, adding the heights of features such as buildings and trees improved the differentiation between vegetation specifically where plants were dense.

Method for Obtaining Phenotypic Traits of Weeds in Complex Field Environments Based on Instance Segmentation

“…To better meet the problem of multiclass imbalance in this paper, we averaged the seven classification indicators [50]. To better evaluate the model algorithm, the IOU is used to conduct measurements [51].…”

Multiple-Oriented and Small Object Detection with Convolutional Neural Networks for Aerial Image

“…The detection of objects in aerial images has been extensively studied over the last few decades, although most works [46][47][48][49][50][51][52][53][54][55][56] focused on extracting the discriminative feature and generating accurate region proposals. Xu et al [46] employed the Viola-Jones approach, the histogram of oriented gradients (HOG) features and the linear support vector machine (SVM) to design a model for vehicle detection in UAV imagery, which exhibited a high performance.…”

“…Furthermore, a CNN model can be used for not only feature extraction but also the generation and classification of the candidate regions. Yan et al [55] proposed a method that used the adaptive intersection-over-union (IoU) information to guide the detection of small-sized objects in aerial imagery. In addition, they designed a type of IoU-based weighted loss, which further improved the detection accuracy.…”

“…Deng et al [50] improved the RPN and Fast R-CNN and proposed a two-way CNN model to realize small-sized object detection. Although the above CNN-based detection models attempted to address the problem of small-sized object detection and could demonstrate satisfactory performance for aerial images, the orientation information was always ignored in these studies [49,50,55] as the RPN-based models can generate only horizontal regions. In contrast, the oriented region proposals are more similar to the real conditions and can help improve the detection accuracy.…”

“…Here, r represents the recall, and p represents the precision [55], which is defined in Equation (9). FP in Equation (9) refers to the false positive.…”