A data science challenge for converting airborne remote sensing data into ecological information

Marconi, Sergio; Graves, Sarah J.; Gong, Dihong; Nia, Morteza Shahriari; Bras, Marion Le; Dorr, Bonnie J.; Fontana, Peter; Gearhart, Justin; Greenberg, Craig S.; Harris, David J.; Kumar, Sugumar Arvind; Nishant, Agarwal; Prarabdh, Joshi; Rege, Sundeep U.; Bohlman, Stephanie A.; White, Ethan; Wang, Daisy Zhe

doi:10.7287/peerj.preprints.26966v1

Cited by 18 publications

(62 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When 32 new methods are introduced in the literature there is often a lack of robust comparison to existing 33 methods, and the comparisons which are included are difficult to apply broadly due to these inherent differences in performance on different systems (4). As well, the formats in which remote sensing data are 35 saved and processed vary hugely across platforms and research disciplines, and have proven difficult to 36 standardize(4).…”

mentioning

confidence: 99%

“…Extracting useful ecosystem parameters from this mass of generated data involves three primary 23 steps: segmentation, alignment, and classification (4). In the segmentation step, individual tree crowns 24 are automatically extracted from the scene so that they can be counted and analyzed separately.…”

mentioning

confidence: 99%

“…The NIST DSE Plant Identification Challenge (4) was introduced this year to try to combat these 38 problems and to increase standardization, methods benchmarking, and collaboration within the remote 39 sensing research community. During this competition, the three tasks outlined above (segmentation, 40 alignment, and classification) were performed on the same dataset by multiple competing teams.…”

mentioning

confidence: 99%

See 2 more Smart Citations

NEON NIST data science evaluation challenge: methods and results of team Conor

McMahon

2018

Preprint

View full text Add to dashboard Cite

The NIST DSE Plant Identification challenge is a new periodic competition focused on improving and generalizing remote sensing processing methods for forest landscapes. To compete in the competition, I created a pipeline to perform three remote sensing tasks.First, a NDVI-and height-thresholded watershed segmentation was performed to identify individual tree crowns using LIDAR height measurements. Second, remote sensing data for segmented crowns was aligned with ground measurements by choosing the set of pairings which minimized error in position and in crown area as predicted by stem height. Third, species classification was performed by reducing the dataset's dimensionality through PCA and then constructing a set of maximum likelihood classifiers to estimate species likelihoods for each tree. Of the three algorithms, the classification routine exhibited the strongest relative performance, with the segmentation algorithm performing the least well. The NIST DSE Plant Identification challenge is a new periodic competition focused on improving and generalizing remote sensing processing methods for forest landscapes. To compete in the competition, I created a pipeline to perform three remote sensing tasks. First, an NDVI-and height-thresholded watershed segmentation was performed to identify individual tree crowns using LIDAR height measurements. Second, remote sensing data for segmented crowns was aligned with ground measurements by choosing the set of pairings which minimized error in position and in crown area as predicted by stem height. Third, species classification was performed by reducing the dataset's dimensionality through PCA and then constructing a set of maximum likelihood classifiers to estimate species likelihoods for each tree. Of the three algorithms, the classification routine exhibited the strongest relative performance, with the segmentation algorithm performing the least well.

show abstract

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

NEON NIST data science evaluation challenge: methods and results of team Conor

McMahon

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Low log loss scores indicate that misclassifications occur at rates close to the rates 178 predicted by the reported probabilities. During model testing, performance was assessed using179 rank-1 accuracy and cross entropy cost(Marconi et al, 2018). Rank-1 accuracy was calculated 180 based on which species ID was predicted with the highest probability.…”

mentioning

confidence: 99%

Untitled

Supplemental Information 1: CCB-ID Prediction Probability Performance

View full text Add to dashboard Cite

Background. Biogeographers assess how species distributions and abundances affect the structure, function, and composition of ecosystems. Yet we face a major challenge: it is difficult to precisely map species across landscapes. Novel Earth observations could obviate this challenge. Airborne imaging spectrometers measure plant functional traits at high resolution, and these measurements can be used to identify tree species. Plant traits are often highly conserved within species, and highly variable between species, which provides the biophysical basis for species mapping. In this paper I describe a trait-based approach to species identification with imaging spectroscopy, CCB-ID, which was developed as part of a NIST-sponsored ecological data science evaluation (ECODSE).Methods. These methods were developed using NEON airborne imaging spectroscopy data. CCB-ID classifies tree species using trait-based reflectance variation and decision tree-based machine learning models, approximating a morphological trait and dichotomous key method traditionally used in botanical classification. First, outliers were removed using a spectral variance threshold. The remaining samples were transformed using principal components analysis and resampled by species to reduce common species biases. Gradient boosting and random forest classifiers were trained using the transformed and resampled feature data. Prediction probabilities were then calibrated using sigmoid regression, and sample-scale predictions were averaged to the crown scale.Results. This approach performed well according to the competition metrics, receiving a rank-1 accuracy score of 0.919, and a cross-entropy cost score of 0.447 on the test data. Accuracy and specificity scores were high for all species, but precision and recall scores were variable for rare species. PCA transformation improved accuracy scores compared to models trained using reflectance data, but outlier removal and data resampling exacerbated class imbalance problems.Discussion. CCB-ID accurately classified tree species using NEON imaging spectroscopy data, reporting the best classification scores among participants. However, it failed to overcome several well-known species mapping challenges, like precisely identifying rare species. Key takeaways include (1) training models to maximize metrics beyond accuracy (e.g. recall) could improve rare species predictions, (2) within-genus trait variation may drive spectral separability, precluding efforts to distinguish between functionally convergent species, (3) outlier removal and data resampling exacerbated class imbalance problems, and should be carefully implemented, (4) PCA transformation greatly improved model results, and (5) feature selection could further improve species classification models. CCB-ID is open source, designed for use with NEON data, and available to support future species mapping efforts.PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.26972v1 | CC BY 4.0 Open Access | rec: AbstractBackground. Biogeographers assess how specie...

show abstract

“…Low log loss scores indicate that misclassifications occur at rates close to the rates 178 predicted by the reported probabilities. During model testing, performance was assessed using 179 rank-1 accuracy and cross entropy cost (Marconi et al, 2018). Rank-1 accuracy was calculated 180 based on which species ID was predicted with the highest probability.…”

mentioning

confidence: 99%

Untitled

Supplemental Information 2: Confusion Matrix of Prediction Probability Results

View full text Add to dashboard Cite

Background. Biogeographers assess how species distributions and abundances affect the structure, function, and composition of ecosystems. Yet we face a major challenge: it is difficult to precisely map species across landscapes. Novel Earth observations could obviate this challenge. Airborne imaging spectrometers measure plant functional traits at high resolution, and these measurements can be used to identify tree species. Plant traits are often highly conserved within species, and highly variable between species, which provides the biophysical basis for species mapping. In this paper I describe a trait-based approach to species identification with imaging spectroscopy, CCB-ID, which was developed as part of a NIST-sponsored ecological data science evaluation (ECODSE).Methods. These methods were developed using NEON airborne imaging spectroscopy data. CCB-ID classifies tree species using trait-based reflectance variation and decision tree-based machine learning models, approximating a morphological trait and dichotomous key method traditionally used in botanical classification. First, outliers were removed using a spectral variance threshold. The remaining samples were transformed using principal components analysis and resampled by species to reduce common species biases. Gradient boosting and random forest classifiers were trained using the transformed and resampled feature data. Prediction probabilities were then calibrated using sigmoid regression, and sample-scale predictions were averaged to the crown scale.Results. This approach performed well according to the competition metrics, receiving a rank-1 accuracy score of 0.919, and a cross-entropy cost score of 0.447 on the test data. Accuracy and specificity scores were high for all species, but precision and recall scores were variable for rare species. PCA transformation improved accuracy scores compared to models trained using reflectance data, but outlier removal and data resampling exacerbated class imbalance problems.Discussion. CCB-ID accurately classified tree species using NEON imaging spectroscopy data, reporting the best classification scores among participants. However, it failed to overcome several well-known species mapping challenges, like precisely identifying rare species. Key takeaways include (1) training models to maximize metrics beyond accuracy (e.g. recall) could improve rare species predictions, (2) within-genus trait variation may drive spectral separability, precluding efforts to distinguish between functionally convergent species, (3) outlier removal and data resampling exacerbated class imbalance problems, and should be carefully implemented, (4) PCA transformation greatly improved model results, and (5) AbstractBackground. Biogeographers assess how species distributions and abundances affect the structure, function, and composition of ecosystems. Yet we face a major challenge: it is difficult to precisely map species across landscapes. Novel Earth observations could obviate this challenge. Airborne imaging spectrometers measure...

show abstract

A data science challenge for converting airborne remote sensing data into ecological information

Cited by 18 publications

References 20 publications

NEON NIST data science evaluation challenge: methods and results of team Conor

NEON NIST data science evaluation challenge: methods and results of team Conor

Untitled

Untitled

Contact Info

Product

Resources

About