From Google Maps to a fine-grained catalog of street trees

Abstract

Up-to-date catalogs of the urban tree population are of importance for municipalities to monitor and improve quality of life in cities. Despite much research on automation of tree mapping, mainly relying on on dedicated airborne LiDAR or hyperspectral campaigns, tree detection and species recognition is still mostly done manually in practice. We present a fully automated tree detection and species recognition pipeline that can process thousands of trees within a few hours using publicly available aerial and street view images of Google Maps TM . These data provide rich information from different viewpoints and at different scales from global tree shapes to bark textures. Our work-flow is built around a supervised classification that automatically learns the most discriminative features from thousands of trees and corresponding, publicly available tree inventory data. In addition, we introduce a change tracker that recognizes changes of individual trees at city-scale, which is essential to keep an urban tree inventory up-to-date. The system takes street-level images of the same tree location at two different times and classifies the type of change (e.g., tree has been removed). Drawing on recent advances in computer vision and machine learning, we apply convolutional neural networks (CNN) for all classification tasks. We propose the following pipeline: download all available panoramas and overhead images of an area of interest, detect trees per image and combine multi-view detections in a probabilistic framework, adding prior knowledge; recognize fine-grained species of detected trees. In a later, separate module, track trees over time, detect significant changes and classify the type of change. We believe this is the first work to exploit publicly available image data for city-scale street tree detection, species recognition and change tracking, exhaustively over several square kilometers, respectively many thousands of trees. Experiments in the city of Pasadena, California, USA show that we can detect > 70% of the street trees, assign correct species to > 80% for 40 different species, and correctly detect and classify changes in > 90% of the cases.

Geocoding of trees from street addresses and street-level images

“…For completeness we note that there is a lot more research on vegetation species mapping but we are not dealing with it in this work and thus a full review is beyond the scope of this paper. We kindly refer the reader to (Alonzo et al, 2014;Wegner et al, 2016;Liu et al, 2017;Branson et al, 2018;Hartling et al;Aval et al;Sidike et al, 2019) for some of the most recent works.…”

“…For tree detection, we download Google street-view panoramas within a 50 meter radius around each address coordinate, detect all trees in the panoramas, project each individual detection to geo-coordinates and combine all individual, geocoded detections per tree into a single geographic location. Trees in street-view panorama images are detected with the Faster R-CNN method (Ren et al, 2015), an object detector using deep convolutional neural networks which we already applied successfully in our recent works (Wegner et al, 2016;Branson et al, 2018). The output of the tree detector is one bounding box per detected tree in each street-view panorama (see example in Fig.…”

“…2). We project all bounding boxes of detected trees per panorama to geographic coordinates assuming locally flat terrain with the equations given in Wegner et al (2016) and Branson et al (2018). More precisely, we assume that the terrain is locally flat and that the camera is levelled against the ground (pitch = 0).…”

An ultrasonic method for 3D reconstruction of surface topography

“…Recently, 3D printing is used to produce objects of almost any geometry or shape from digital data or electronic data source. A few companies such as Google and Microsoft enabled their hardware to perform 3D scanning [28,29]. In this work it has been carried out that the ultrasonic waves can be applied as an option of 3D scanning reconstruction.…”

Defoliation estimation of forest trees from ground-level images

“…Another option, which we follow in our work presented in this paper, is analysis of photos acquired on the ground (Du et al, 2007;Kumar et al, 2012;Mouine et al, 2013;Goëau et al, 2013Goëau et al, , 2014Wegner et al, 2016;Branson et al, 2018). Ground-level images have higher spatial resolution than remote sensing data and provide a horizontal view on the entire tree comparable to the view of the expert assessing the tree visually in the field (Solberg and Strand, 1999).…”