Semantic Segmentation of Tree-Canopy in Urban Environment with Pixel-Wise Deep Learning

Abstract

Urban forests are an important part of any city, given that they provide several environmental benefits, such as improving urban drainage, climate regulation, public health, biodiversity, and others. However, tree detection in cities is challenging, given the irregular shape, size, occlusion, and complexity of urban areas. With the advance of environmental technologies, deep learning segmentation mapping methods can map urban forests accurately. We applied a region-based CNN object instance segmentation algorithm for the semantic segmentation of tree canopies in urban environments based on aerial RGB imagery. To the best of our knowledge, no study investigated the performance of deep learning-based methods for segmentation tasks inside the Cerrado biome, specifically for urban tree segmentation. Five state-of-the-art architectures were evaluated, namely: Fully Convolutional Network; U-Net; SegNet; Dynamic Dilated Convolution Network and DeepLabV3+. The experimental analysis showed the effectiveness of these methods reporting results such as pixel accuracy of 96,35%, an average accuracy of 91.25%, F1-score of 91.40%, Kappa of 82.80% and IoU of 73.89%. We also determined the inference time needed per area, and the deep learning methods investigated after the training proved to be suitable to solve this task, providing fast and effective solutions with inference time varying from 0.042 to 0.153 minutes per hectare. We conclude that the semantic segmentation of trees inside urban environments is highly achievable with deep neural networks. This information could be of high importance to decision-making and may contribute to the management of urban systems. It should be also important to mention that the dataset used in this work is available on our website.

Identifying plant species in kettle holes using UAV images and deep learning techniques

“…Remote sensing in combination with machine learning technologies can provide a potentially practical and cost‐effective way to study changes in vegetation cover. Results of this study utilizing machine learning methods to UAV images achieved F 1‐scores that are comparable to previous studies of plant segmentation (Elkind et al., 2019; Martins et al., 2021; Torres et al., 2020). Some studies implemented additional information such as digital surface models (DSMs) or multi‐spectral data to classify species to obtain similar or better F 1‐scores (Benjamin et al., 2021; Chabot et al., 2018; Durgan et al., 2020; Husson et al., 2016; Schulze‐Brüninghoff et al., 2021).…”

“…These techniques can adapt to changing environments and enable ongoing improvement to understand the scene. With the advancement of mapping and computer technologies, DL methods are being tested to identify patterns in the most varied fields of science (Dos Santos Ferreira et al., 2017; Martins et al., 2021; Shen et al., 2017; Torres et al., 2020). Thus far, a large amount of training data are needed to train a DL‐based network for subsequent classification (Goodfellow et al., 2016).…”

Predicting urban tree cover from incomplete point labels and limited background information

“…Second, we focus on urban areas, which are relatively under-explored in other work [16,29], although many free data sources exist. Third, existing work relies on strong preprocessing and fully annotated data in which the object has either been accurately delineated [7,20] or been annotated by a bounding box or at least a point label. An example of point labels is done by Ventura et al [30], who manually annotated 100 000 trees from eight cities in the USA and collected multiple years of imagery.…”

Canopy Volume Extraction of Citrus reticulate Blanco cv. Shatangju Trees Using UAV Image-Based Point Cloud Deep Learning

“…For the segmentation of 2D (Two-Dimensional) images, scholars have tended to use a convolutional neural network in deep learning [16][17][18] from traditional segmentation methods based on edge detection, threshold, region and specific theoretical tools. For example, Martins et al [19] segmented the trees in the urban environment image, Yan et al [20] identified different tree species and Chadwick et al [21] extracted the height of the crown of a single coniferous tree. However, 2D images have problems such as insufficient spatial information and occlusion.…”