Individual Tree-Crown Detection in RGB Imagery Using Semi-Supervised Deep Learning Neural Networks

Abstract

Remote sensing can transform the speed, scale, and cost of biodiversity and forestry surveys. Data acquisition currently outpaces the ability to identify individual organisms in high resolution imagery. We outline an approach for identifying tree-crowns in RGB imagery while using a semi-supervised deep learning detection network. Individual crown delineation has been a long-standing challenge in remote sensing and available algorithms produce mixed results. We show that deep learning models can leverage existing Light Detection and Ranging (LIDAR)-based unsupervised delineation to generate trees that are used for training an initial RGB crown detection model. Despite limitations in the original unsupervised detection approach, this noisy training data may contain information from which the neural network can learn initial tree features. We then refine the initial model using a small number of higher-quality hand-annotated RGB images. We validate our proposed approach while using an open-canopy site in the National Ecological Observation Network. Our results show that a model using 434,551 self-generated trees with the addition of 2848 hand-annotated trees yields accurate predictions in natural landscapes. Using an intersection-over-union threshold of 0.5, the full model had an average tree crown recall of 0.69, with a precision of 0.61 for the visually-annotated data. The model had an average tree detection rate of 0.82 for the field collected stems. The addition of a small number of hand-annotated trees improved the performance over the initial self-supervised model. This semi-supervised deep learning approach demonstrates that remote sensing can overcome a lack of labeled training data by generating noisy data for initial training using unsupervised methods and retraining the resulting models with high quality labeled data.

Geographic Generalization in Airborne RGB Deep Learning Tree Detection

“…We used our previously developed self-supervised algorithm for RGB-based tree identification (Weinstein et al 2019). This method uses the Retinanet one-stage object detector (Gaiser et al 2017) with a Resnet-50 classification backbone, which allows pixel information to be shared at multiple scales, from individual pixels to groups of connected objects.…”

“…We then retrained the network using the hand-annotated data for 40 epochs. For more details of this semi-supervised approach see Weinstein et al (2019).…”

“…Weinstein (et al 2019) recently developed a deep learning approach for tree detection using RGB data that has the potential to address these requirements for identifying trees across forest types. The semi-supervised method first uses unsupervised LIDAR-based tree detection (e.g., Silva et al 2016) to generate millions of labeled trees that are used to pretraining of the neural network.…”

“…Here we build on Weinstein et al (2019) explore the potential of this tree detection method to generalize across sites by evaluating its performance on a range of forest types, assessing the transferability of tree features across forest types, and exploring the possibility of building a single unified tree detection model. Our aim is to test a deep learning approach 1) for identifying trees in four different forest types when trained on that forest type (‘within-site’); 2) for identifying trees when trained on data from other forest types (‘cross-site’); 3) for combining pretraining data from other sites with hand-annotated data from a new site (‘transfer learning’); and 4) for comparing the performance of a within-site model to a universal model fit to data on all forest types simultaneously (‘universal’).…”

Leveraging Environmental Research and Observation Networks to Advance Soil Carbon Science

“…Lastly, the networks should be test beds for developing new tools, data collection techniques, and models that are particularly promising for improved understanding of SOM dynamics. Examples include the increasing popularity of multiscale geophysical techniques for investigating the shallow subsurface in the CZO network (Parsekian et al, ) and the broad application of airborne LiDAR and hyperspectral remote sensing techniques at NEON (Weinstein et al, ). Moreover, there are likely to be data‐rich nodes within the networks that provide opportunities to prototype cross‐disciplinary syntheses.…”