Deep learning object detection to estimate the nectar sugar mass of flowering vegetation

Abstract

Floral resources are a key driver of pollinator abundance and diversity, yet their quantification in the field and laboratory is laborious and requires specialist skills. Using a dataset of 25,000 labelled tags of fieldwork‐realistic quality, a convolutional neural network (Faster R‐CNN) was trained to detect the nectar‐producing floral units of 25 taxa in surveyors’ quadrat images of native, weed‐rich grassland in the United Kingdom. Floral unit detection on a test set of 50 model‐unseen images of comparable vegetation returned a precision of 90%, recall of 86% and F1 score (the harmonic mean of precision and recall) of 88%. Model performance was consistent across the range of floral abundance in this habitat. Comparison of the nectar sugar mass estimates made by the CNN and three human surveyors returned similar means and standard deviations. Over half of the nectar sugar mass estimates made by the model fell within the absolute range of those of the human surveyors. The optimal number of quadrat image samples was determined to be the same for the CNN as for the average human surveyor. For a standard quadrat sampling protocol of 10–15 replicates, this application of deep learning could cut pollinator‐plant survey time per stand of vegetation from hours to minutes. The CNN is restricted to a single view of a quadrat, with no scope for manual examination or specimen collection, though in contrast to human surveyors its object detection is deterministic and its floral unit definition is standardized. As agri‐environment schemes move from prescriptive to results‐based, this approach provides an independent barometer of grassland management which is usable by both landowner and scheme administrator. The model can be adapted to visual estimations of other ecological resources such as winter bird food, floral pollen volume, insect infestation and tree flowering/fruiting, and by adjustment of classification threshold may show acceptable taxonomic differentiation for presence–absence surveys.

Data-centric AI approach for automated wildflower monitoring

“…This high score makes the knowledge encapsulated by model actionable, i.e., it can be applied in practice for automatic large-scale wildflower monitoring. Other F-RCNN object detection models for wildflower monitoring are published by Hicks et al [39] and Gallman et al [60]. However, these studies, having 25K and 10K annotations respectively, collected wildflower data in limited period of one flowering season (May till August) and their models are able to identify and count only 25 flowering plant species, half the number of species that our model can handle.…”

“…Both support the monitoring use case, i.e., the use of object detection models that can classify and localize . The Nectar dataset [39] consists of nearly 2000 Canon Powershot G10 (14.7 Mpixels) images of flowering plant species collected in one flowering season in one habitat type, viz. weed-rich grasslands in the UK.…”

“…All 25 species are easy to outline and have predominantly simple inflorescences, i.e., solitary flowers or flower heads. The aim of the Hicks et al study [39] was to estimate nectar sugar mass as an indicator for effectiveness of nature-inclusive farming. In other words, these authors used wildflower monitoring as a means to an end.…”

Defining Quality Requirements for a Trustworthy AI Wildflower Monitoring Platform

“…Object detection models are trained with images containing tagged bounding boxes around objects of interest. Hicks et al [20] presented a first object detection model that was able to count 25 different wildflower species in images. In FlowerPower, the concept of Hicks et al [20] is expanded to a more advanced data-centric object detection model that can be used to count 100+ different wildflower species and a comprehensive software platform that supports various enduser scenarios.…”

Toward evidence-based decision support systems to optimize pollination and yields in highbush blueberry

“…Empirically, Reilly et al (2020) found that "Bluecrop" fields would require 16.7-26.3 bees per 100 m row to maximize fruit weight. Pollinator density estimates should always be associated with a flower count in pollination empirical studies (Chabert et al, 2022), highlighting the importance of developing simple and fast flower monitoring approaches with new methods such as image analysis with deep learning (Farjon et al, 2020;Hicks et al, 2021). Flower monitoring approaches could also be used by growers and crop advisors to assess pollination and inform pollination management.…”