ARBOR: A new framework for assessing the accuracy of individual tree crown delineation from remotely-sensed data

Abstract

To assess the accuracy of individual tree crown (ITC) delineation techniques the same tree needs to be identified in two different datasets, for example, ground reference (GR) data and crowns delineated from LiDAR. Many studies use arbitrary metrics or simple linear-distance thresholds to match trees in different datasets without quantifying the level of agreement. For example, successful match-pairing is often claimed where two data points, representing the same tree in different datasets, are located within 5m of one another. Such simple measures are inadequate for representing the multi-variate nature of ITC delineations and generate misleading measures of delineation accuracy. In this study, we develop a new framework for objectively quantifying the agreement between GR and remotely-sensed tree datasets: the Accuracy of Remotely-sensed Biophysical Observation and Retrieval (ARBOR) framework. Using common biophysical properties of ITC delineated trees (location, height and crown area), trees represented in different data sets were modelled as overlapping Gaussian curves to facilitate a more comprehensive assessment of the level of agreement. Extensive testing quantified the limitations of some frequently used match-pairing methods, in particular, the Hausdorff distance algorithm. We demonstrate that within the ARBOR framework, the Hungarian combinatorial optimisation algorithm improves the match between datasets, while the Jaccard similarity coefficient is effective for measuring the correspondence between the matched data populations. The ARBOR framework was applied to GR and remotely-sensed tree data from a woodland study site to demonstrate how ARBOR can identify the optimum ITC delineation technique, out of four different methods tested, based on two measures of statistical accuracy. Using ARBOR will limit further reliance on arbitrary thresholds as it provides an objective approach for quantifying accuracy in the development and application of ITC delineation algorithms.

Individual-Tree Segmentation from UAV–LiDAR Data Using a Region-Growing Segmentation and Supervoxel-Weighted Fuzzy Clustering Approach

“…Therefore, an ARBOR method was used in this study to quantify the agreement between the reference tree datasets and detected trees. The framework was proposed by Murray et al [41] and models trees as Gaussian curves with the use of biophysical properties (tree location, tree height, and crown area). A Jaccard similarity coefficient was then developed to assess the agreements of the trees in different datasets as follows:…”

“…In addition, instead of the simple universal location and linear-distance measurements used in most studies, in this study, a new "accuracy of remotely sensed biophysical observation and retrieval" (ARBOR) framework was introduced. ARBOR matches trees with various metrics (tree height, location, and crown area) to quantify the accuracy of the segmentation results [41].…”

Evaluation of the NEON dataset for the automatic detection of treetops and delineation of tree crowns in American and German mixed forests

“…Either a traditional method has been utilised, 8, 50, 58 a statistical measure has slightly been adjusted and renamed, 59 or a new method has been defined. 60 This inconsistency of evaluation brings challenges for the comparison across different detection or delineation studies, 7 which is further complicated by other factors like the datasets' spatial resolution and the forest structure that affect the methods' accuracy. 56 Besides a visual assessment, it has been decided to use only traditional metrics with respect to the Ground Truth data, namely the confusion matrix in combination with the F1-Score 61 and Intersection over Union (IoU).…”