Capturing long-tailed individual tree diversity using an airborne multi-temporal hierarchical model

Abstract

Measuring forest biodiversity using terrestrial surveys is expensive and can only capture common species abundance in large heterogeneous landscapes. In contrast, combining airborne imagery with computer vision can generate individual tree data at the scales of hundreds of thousands of trees. To train computer vision models, ground-based species labels are combined with airborne reflectance data. Due to the difficulty of finding rare species in a large landscape, the majority of classification models only include the most abundant species, leading to biased predictions at broad scales. Extending classification models to include rare species requires targeted data collection and algorithmic improvements to overcome large data imbalances between dominant and rare taxa. In addition, large landscapes often require multiple acquisition events, leading to significant within-species variation in reflectance spectra. Using a multi-temporal hierarchical model, we demonstrate the ability to include species predicted at less than 1 frequency in landscape without losing performance on the dominant species. The final model has over 75% accuracy for 14 species with improved rare species classification compared to a baseline deep learning model. After filtering out dead trees, we generate landscape species maps of individual crowns for over 670,000 individual trees at the Ordway Swisher Biological Station within the National Ecological Observatory Network. We estimate the relative abundance of the species within the landscape and provide three measures of uncertainty to generate a range of counts for each species. These maps provide the first estimates of canopy tree diversity within NEON sites to include rare species and provide a blueprint for capturing tree diversity using airborne computer vision at broad scales.

Individual canopy tree species maps for the National Ecological Observatory Network

“…This label should be interpreted as provisional since trees can lose leaves due to a variety of causes such as insect defoliation in one year, but ultimately recover over time. Presented in Weinstein et al (2023), this Alive-Dead model is a two class resnet-50 deep learning neural network trained on hand-annotated images from across all NEON sites. During prediction, the location of each predicted crown is cropped and passed to the Alive-Dead model for labeling as Alive (0) or Dead (1) with a confidence score for each class.…”

“…To classify each tree crown to species, we use a multi-temporal hierarchical model (Weinstein et al 2023). The predicted species label confidence score, as well as labels from the higher taxonomic levels, are included in the shapefile (Table 1).…”

“…Here, we combine airborne RGB, hyperspectral, and LiDAR data, to predict 100 million tree locations for 81 species within 24 NEON sites across the United States. Adapting the workflows developed from our research on predicting individual tree species identity from remote sensing (Marconi et al 2022, Weinstein et al 2023), we use a series of machine learning models to generate crown position, species label, health status and height on individual trees visible in the canopy. We make these data openly available as 1km tiles to foster research in forest ecology, as well as regional scale ecological remote sensing applications.…”

“…Dozens of models have been proposed, but it is unclear if they are successful when applied to a variety of ecosystems with differences in tree density, abundance distributions, and spectral backgrounds. Our goal was to apply the multi-temporal hierarchical model proposed in Weinstein et al 2023 to sites across the United States, with a diverse set of forests including conifer, mixed hardwoods, closed canopy broadleaf and open woodlands. This model provides two key improvements over standard deep learning approaches.…”

“…Conceptual workflow for species prediction at each NEON site. The figure is modified from Weinstein et al 2023.…”